2024-04-15T15:03:35.279 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-15T15:03:35.280 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_leonidus@teuthology --name leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/quiesce} --archive /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119 --verbose -- /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/orig.config.yaml 2024-04-15T15:03:35.327 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-15T15:03:35.418 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119 branch: wip-lusov-quiesce-xlock description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/quiesce} email: null first_in_suite: false job_id: '7657119' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce-xlock ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow request sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce-xlock sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2288 sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 sleep_before_teardown: 0 suite: fs suite_branch: wip-lusov-quiesce-xlock suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDB2/YdZYsSxPRvJ82wegIo13jQ1h3vGBQsXIodUm9Dh1Wb3P75Jv3mchNNDhv2hcIMgigAsLdPTiTAdgMChKRA= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJQqSxuRx5wTNkPdbKM8TVcVuTeoPB/CT9UuQMYOsso+CKgHzs0p5W4GWzB0DfADwhDMn8308WtVk0DNgnQRUI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_quiesce teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-15_14:46:31 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-15T15:03:35.418 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa; will attempt to use it 2024-04-15T15:03:35.419 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks 2024-04-15T15:03:35.419 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-15T15:03:35.421 INFO:teuthology.task.internal:Checking packages... 2024-04-15T15:03:35.447 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14' 2024-04-15T15:03:35.447 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-15T15:03:35.447 INFO:teuthology.packaging:ref: None 2024-04-15T15:03:35.447 INFO:teuthology.packaging:tag: None 2024-04-15T15:03:35.447 INFO:teuthology.packaging:branch: wip-lusov-quiesce-xlock 2024-04-15T15:03:35.447 INFO:teuthology.packaging:sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:03:35.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce-xlock 2024-04-15T15:03:35.592 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2861.g9bf1c4ba 2024-04-15T15:03:35.594 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-15T15:03:35.600 INFO:teuthology.task.internal:no buildpackages task found 2024-04-15T15:03:35.600 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-15T15:03:35.606 INFO:teuthology.task.internal:Saving configuration 2024-04-15T15:03:35.624 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-15T15:03:35.630 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-15T15:03:35.657 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 14:56:56.578553', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDB2/YdZYsSxPRvJ82wegIo13jQ1h3vGBQsXIodUm9Dh1Wb3P75Jv3mchNNDhv2hcIMgigAsLdPTiTAdgMChKRA='} 2024-04-15T15:03:35.676 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 14:56:56.577236', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJQqSxuRx5wTNkPdbKM8TVcVuTeoPB/CT9UuQMYOsso+CKgHzs0p5W4GWzB0DfADwhDMn8308WtVk0DNgnQRUI='} 2024-04-15T15:03:35.676 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-15T15:03:35.683 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-15T15:03:35.683 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-15T15:03:35.683 INFO:teuthology.run_tasks:Running task console_log... 2024-04-15T15:03:35.761 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9d393100d0>, signals=[15]) 2024-04-15T15:03:35.762 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-15T15:03:35.768 INFO:teuthology.task.internal:Opening connections... 2024-04-15T15:03:35.768 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:03:35.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:03:35.878 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:03:35.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:03:35.966 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-15T15:03:35.973 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-04-15T15:03:35.991 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-04-15T15:03:35.991 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-04-15T15:03:36.047 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-15T15:03:36.048 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-15T15:03:36.049 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-15T15:03:36.049 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-04-15T15:03:36.074 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-04-15T15:03:36.091 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-04-15T15:03:36.091 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-04-15T15:03:36.147 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-04-15T15:03:36.147 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-04-15T15:03:36.147 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-04-15T15:03:36.147 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-04-15T15:03:36.147 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-04-15T15:03:36.148 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-04-15T15:03:36.148 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-15T15:03:36.148 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-04-15T15:03:36.148 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-04-15T15:03:36.148 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-15T15:03:36.148 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-04-15T15:03:36.148 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-15T15:03:36.148 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-15T15:03:36.148 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-15T15:03:36.148 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-04-15T15:03:36.165 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-15T15:03:36.175 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-15T15:03:36.181 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-15T15:03:36.182 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-04-15T15:03:36.184 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-04-15T15:03:36.203 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-15T15:03:36.214 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-15T15:03:36.214 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-04-15T15:03:36.241 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-04-15T15:03:36.347 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-15T15:03:36.396 INFO:teuthology.run_tasks:Running task kernel... 2024-04-15T15:03:36.411 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-15T15:03:36.411 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-15T15:03:36.411 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-15T15:03:36.412 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-15T15:03:36.412 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-15T15:03:36.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:03:36.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-15T15:03:36.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:03:36.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-15T15:03:36.768 DEBUG:teuthology.task.kernel:sha1 for client.1 is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:03:36.768 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-15T15:03:36.768 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-04-15T15:03:36.776 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:03:36.776 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-15T15:03:36.776 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-04-15T15:03:36.786 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-361.el9.x86_64 2024-04-15T15:03:36.786 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:03:36.787 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-15T15:03:36.787 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum upgrade -y linux-firmware 2024-04-15T15:03:36.795 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-361.el9.x86_64 2024-04-15T15:03:36.795 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:03:36.795 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-15T15:03:36.795 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum upgrade -y linux-firmware 2024-04-15T15:03:38.901 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-15T15:03:39.028 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-15T15:03:40.327 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2024-04-15T15:03:40.495 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-04-15T15:03:41.428 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-15T15:03:49.816 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 899 kB/s | 8.0 MB 00:09 2024-04-15T15:04:01.026 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:18 2024-04-15T15:04:08.726 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.2 MB 00:03 2024-04-15T15:04:10.460 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-15T15:04:12.388 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-15T15:04:12.680 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 889 kB/s | 19 MB 00:21 2024-04-15T15:04:16.754 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-15T15:04:17.595 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-04-15T15:04:19.047 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-15T15:04:20.032 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-04-15T15:04:20.116 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-15T15:04:20.125 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-15T15:04:20.125 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-15T15:04:20.125 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-15T15:04:20.125 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-04-15T15:04:20.125 INFO:teuthology.orchestra.run.smithi049.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-15T15:04:20.125 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:04:20.125 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-15T15:04:20.125 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-15T15:04:20.126 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 1 Package 2024-04-15T15:04:20.126 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:04:20.126 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 375 M 2024-04-15T15:04:20.126 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-15T15:04:25.006 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-15T15:04:27.497 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-15T15:04:27.512 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-15T15:04:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-15T15:04:27.513 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-15T15:04:27.513 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-04-15T15:04:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-15T15:04:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:04:27.513 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-15T15:04:27.513 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-15T15:04:27.513 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 1 Package 2024-04-15T15:04:27.513 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:04:27.513 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 375 M 2024-04-15T15:04:27.514 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-15T15:04:28.570 INFO:teuthology.orchestra.run.smithi049.stdout:linux-firmware-20240219-143.el9.noarch.rpm 44 MB/s | 375 MB 00:08 2024-04-15T15:04:28.571 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:04:28.571 INFO:teuthology.orchestra.run.smithi049.stdout:Total 44 MB/s | 375 MB 00:08 2024-04-15T15:04:28.571 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-15T15:04:28.663 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-15T15:04:28.663 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-15T15:04:31.173 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-15T15:04:31.174 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-15T15:04:37.573 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-15T15:04:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-15T15:04:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:04:38.189 INFO:teuthology.orchestra.run.smithi107.stdout:linux-firmware-20240219-143.el9.noarch.rpm 35 MB/s | 375 MB 00:10 2024-04-15T15:04:38.189 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:04:38.190 INFO:teuthology.orchestra.run.smithi107.stdout:Total 35 MB/s | 375 MB 00:10 2024-04-15T15:04:38.190 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-15T15:04:38.292 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-15T15:04:38.292 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-15T15:04:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:04:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-15T15:04:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:04:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:04:39.780 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-04-15T15:04:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-15T15:04:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:04:39.780 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-15T15:04:40.084 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-15T15:04:40.084 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:04:40.308 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-15T15:04:40.308 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-15T15:04:40.311 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi049.front.sepia.ceph.com to complete... 2024-04-15T15:04:40.762 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-15T15:04:40.762 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-15T15:04:47.043 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-15T15:04:47.255 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-15T15:04:47.338 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:04:48.774 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:04:48.774 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-15T15:04:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:04:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:04:49.170 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-04-15T15:04:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-15T15:04:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:04:49.170 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-15T15:04:49.492 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.1... 2024-04-15T15:04:49.493 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:04:49.636 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-15T15:04:49.636 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-15T15:04:49.639 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi107.front.sepia.ceph.com to complete... 2024-04-15T15:04:55.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15 15:04:55 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-15T15:04:55.387 INFO:teuthology.orchestra.run.smithi049.stderr:FINISHED --2024-04-15 15:04:55-- 2024-04-15T15:04:55.387 INFO:teuthology.orchestra.run.smithi049.stderr:Total wall clock time: 15s 2024-04-15T15:04:55.387 INFO:teuthology.orchestra.run.smithi049.stderr:Downloaded: 1 files, 341M in 15s (23.1 MB/s) 2024-04-15T15:04:55.390 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-15T15:04:55.390 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-15T15:04:57.547 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying... ######################################## 2024-04-15T15:04:57.603 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing... ######################################## 2024-04-15T15:04:57.687 INFO:teuthology.orchestra.run.smithi049.stdout:Updating / installing... 2024-04-15T15:05:05.975 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-15T15:05:22.888 INFO:teuthology.orchestra.run.smithi049.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-15T15:05:46.638 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-15T15:05:46.638 DEBUG:teuthology.orchestra.run.smithi049:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-15T15:05:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:05:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:05:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:05:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-15T15:05:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-15T15:05:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-15T15:05:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-15T15:05:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-15T15:05:46.713 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-15T15:05:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-15T15:05:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-15T15:05:46.716 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-15T15:05:46.716 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-15T15:05:46.716 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-15T15:05:46.716 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-15T15:05:46.716 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-15T15:05:46.716 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-15T15:05:46.716 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-15T15:05:46.716 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-15T15:05:46.717 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-15T15:05:46.718 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-15T15:05:46.719 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-15T15:05:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-15T15:05:46.721 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-15T15:05:46.722 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-15T15:05:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-15T15:05:46.724 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-15T15:05:46.725 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-15T15:05:46.726 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-15T15:05:46.727 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-15T15:05:46.728 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-15T15:05:46.729 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-15T15:05:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-15T15:05:46.731 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-15T15:05:46.732 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-15T15:05:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-15T15:05:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-15T15:05:46.735 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-15T15:05:46.736 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-15T15:05:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-15T15:05:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-15T15:05:46.739 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-15T15:05:46.740 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-15T15:05:46.741 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-15T15:05:46.742 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-15T15:05:46.743 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-15T15:05:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-15T15:05:46.745 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-15T15:05:46.746 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-15T15:05:46.747 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-15T15:05:46.748 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-15T15:05:46.749 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-15T15:05:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-15T15:05:46.751 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-15T15:05:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-15T15:05:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-15T15:05:46.754 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-15T15:05:46.755 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-15T15:05:46.756 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-15T15:05:46.757 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-15T15:05:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-15T15:05:46.759 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-15T15:05:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-15T15:05:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-15T15:05:46.762 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-15T15:05:46.763 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-15T15:05:46.764 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-15T15:05:46.765 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-15T15:05:46.766 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-15T15:05:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-15T15:05:46.768 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-15T15:05:46.769 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-15T15:05:46.770 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-15T15:05:46.771 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-15T15:05:46.772 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-15T15:05:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-15T15:05:46.774 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-15T15:05:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-15T15:05:46.776 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-15T15:05:46.777 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-15T15:05:46.778 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-15T15:05:46.779 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-15T15:05:46.780 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-15T15:05:46.781 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-15T15:05:46.782 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-15T15:05:46.783 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-15T15:05:46.784 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-15T15:05:46.785 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-15T15:05:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-15T15:05:46.787 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-15T15:05:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-15T15:05:46.789 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-15T15:05:46.790 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-15T15:05:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-15T15:05:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-15T15:05:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-15T15:05:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-15T15:05:46.795 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-15T15:05:46.796 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-15T15:05:46.797 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-15T15:05:46.798 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-15T15:05:46.798 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-15T15:05:46.798 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-15T15:05:46.798 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-15T15:05:46.798 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-15T15:05:46.798 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-15T15:05:46.798 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-15T15:05:46.798 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-15T15:05:46.798 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-15T15:05:46.798 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-15T15:05:46.799 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-15T15:05:46.800 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-15T15:05:46.801 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-15T15:05:46.802 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-15T15:05:46.803 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-15T15:05:46.804 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-15T15:05:46.805 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-15T15:05:46.806 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-15T15:05:46.807 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-15T15:05:46.808 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-15T15:05:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-15T15:05:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-15T15:05:46.811 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-15T15:05:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-15T15:05:46.813 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-15T15:05:46.814 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-15T15:05:46.815 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-15T15:05:46.816 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-15T15:05:46.817 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-15T15:05:46.818 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-15T15:05:46.819 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-15T15:05:46.820 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-15T15:05:46.821 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-15T15:05:46.822 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-15T15:05:46.823 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-15T15:05:46.824 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-15T15:05:46.825 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-15T15:05:46.826 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-15T15:05:46.827 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-15T15:05:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-15T15:05:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-15T15:05:46.830 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-15T15:05:46.831 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-15T15:05:46.832 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-15T15:05:46.833 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-15T15:05:46.834 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-15T15:05:46.835 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-15T15:05:46.836 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-15T15:05:46.837 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-15T15:05:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-15T15:05:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-15T15:05:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-15T15:05:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-15T15:05:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-15T15:05:46.843 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-15T15:05:46.844 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-15T15:05:46.845 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-15T15:05:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-15T15:05:46.847 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-15T15:05:46.848 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-15T15:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-15T15:05:46.850 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-15T15:05:46.851 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-15T15:05:46.852 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-15T15:05:46.853 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-15T15:05:46.854 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-15T15:05:46.855 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-15T15:05:46.858 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:05:46.859 DEBUG:teuthology.orchestra.run.smithi049:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-15T15:05:46.883 INFO:teuthology.orchestra.run.smithi049.stdout:postinstall scriptlet (using /bin/sh): 2024-04-15T15:05:46.883 INFO:teuthology.orchestra.run.smithi049.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-15T15:05:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-15T15:05:46.883 INFO:teuthology.orchestra.run.smithi049.stdout:fi 2024-04-15T15:05:46.883 INFO:teuthology.orchestra.run.smithi049.stdout:for file in vmlinuz System.map config; do 2024-04-15T15:05:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: fi 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout:done 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout:fi 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout:fi 2024-04-15T15:05:46.884 INFO:teuthology.orchestra.run.smithi049.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-15T15:05:46.885 INFO:teuthology.orchestra.run.smithi049.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-15T15:05:46.885 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:05:46.885 INFO:teuthology.orchestra.run.smithi049.stdout:fi 2024-04-15T15:05:46.885 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -qi grub2-tools 2024-04-15T15:05:46.962 INFO:teuthology.orchestra.run.smithi049.stdout:Name : grub2-tools 2024-04-15T15:05:46.962 INFO:teuthology.orchestra.run.smithi049.stdout:Epoch : 1 2024-04-15T15:05:46.962 INFO:teuthology.orchestra.run.smithi049.stdout:Version : 2.06 2024-04-15T15:05:46.962 INFO:teuthology.orchestra.run.smithi049.stdout:Release : 68.el9 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: x86_64 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:Group : Unspecified 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:Size : 8264254 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:License : GPLv3+ 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:Packager : builder@centos.org 2024-04-15T15:05:46.963 INFO:teuthology.orchestra.run.smithi049.stdout:Vendor : CentOS 2024-04-15T15:05:46.964 INFO:teuthology.orchestra.run.smithi049.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-15T15:05:46.964 INFO:teuthology.orchestra.run.smithi049.stdout:Summary : Support tools for GRUB. 2024-04-15T15:05:46.964 INFO:teuthology.orchestra.run.smithi049.stdout:Description : 2024-04-15T15:05:46.964 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:05:46.964 INFO:teuthology.orchestra.run.smithi049.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-15T15:05:46.964 INFO:teuthology.orchestra.run.smithi049.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-15T15:05:46.964 INFO:teuthology.orchestra.run.smithi049.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-15T15:05:46.964 INFO:teuthology.orchestra.run.smithi049.stdout:hardware devices. 2024-04-15T15:05:46.964 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:05:46.964 INFO:teuthology.orchestra.run.smithi049.stdout:This subpackage provides tools for support of all platforms. 2024-04-15T15:05:46.965 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-15T15:05:46.966 INFO:teuthology.task.kernel:Updating grub on smithi049 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:05:46.966 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-15T15:05:47.541 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2024-04-15T15:05:48.920 INFO:teuthology.orchestra.run.smithi049.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-15T15:05:48.947 INFO:teuthology.orchestra.run.smithi049.stderr:done 2024-04-15T15:05:48.950 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-04-15T15:05:48.965 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.DHkiPHIwfO 2024-04-15T15:05:48.965 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.DHkiPHIwfO 2024-04-15T15:05:49.043 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.DHkiPHIwfO 2024-04-15T15:05:49.184 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.DHkiPHIwfO is 6KB 2024-04-15T15:05:49.205 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.DHkiPHIwfO 2024-04-15T15:05:49.219 DEBUG:teuthology.orchestra.run.smithi049:> sudo /bin/ls /boot/loader/entries || true 2024-04-15T15:05:49.294 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-15T15:05:49.294 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-15T15:05:49.294 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-15T15:05:49.294 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-15T15:05:49.295 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:05:49.455 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2024-04-15T15:06:00.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15 15:06:00 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-15T15:06:00.427 INFO:teuthology.orchestra.run.smithi107.stderr:FINISHED --2024-04-15 15:06:00-- 2024-04-15T15:06:00.427 INFO:teuthology.orchestra.run.smithi107.stderr:Total wall clock time: 1m 11s 2024-04-15T15:06:00.427 INFO:teuthology.orchestra.run.smithi107.stderr:Downloaded: 1 files, 341M in 1m 10s (4.88 MB/s) 2024-04-15T15:06:00.429 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.1... 2024-04-15T15:06:00.430 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-15T15:06:02.472 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying... ######################################## 2024-04-15T15:06:02.625 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing... ######################################## 2024-04-15T15:06:02.704 INFO:teuthology.orchestra.run.smithi107.stdout:Updating / installing... 2024-04-15T15:06:10.652 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-15T15:06:19.458 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-15T15:06:19.459 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-15T15:06:19.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:06:25.557 INFO:teuthology.orchestra.run.smithi107.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-15T15:06:37.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-04-15T15:06:46.983 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-15T15:06:46.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:06:48.299 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-15T15:06:48.300 DEBUG:teuthology.orchestra.run.smithi107:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-15T15:06:48.378 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:06:48.378 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:06:48.378 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:06:48.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-15T15:06:48.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-15T15:06:48.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-15T15:06:48.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-15T15:06:48.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-15T15:06:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-15T15:06:48.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-15T15:06:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-15T15:06:48.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-15T15:06:48.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-15T15:06:48.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-15T15:06:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-15T15:06:48.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-15T15:06:48.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-15T15:06:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-15T15:06:48.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-15T15:06:48.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-15T15:06:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-15T15:06:48.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-15T15:06:48.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-15T15:06:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-15T15:06:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-15T15:06:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-15T15:06:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-15T15:06:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-15T15:06:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-15T15:06:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-15T15:06:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-15T15:06:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-15T15:06:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-15T15:06:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-15T15:06:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-15T15:06:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-15T15:06:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-15T15:06:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-15T15:06:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-15T15:06:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-15T15:06:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-15T15:06:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-15T15:06:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-15T15:06:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-15T15:06:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-15T15:06:48.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-15T15:06:48.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-15T15:06:48.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-15T15:06:48.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-15T15:06:48.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-15T15:06:48.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-15T15:06:48.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-15T15:06:48.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-15T15:06:48.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-15T15:06:48.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-15T15:06:48.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-15T15:06:48.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-15T15:06:48.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-15T15:06:48.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-15T15:06:48.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-15T15:06:48.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-15T15:06:48.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-15T15:06:48.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-15T15:06:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-15T15:06:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-15T15:06:48.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-15T15:06:48.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-15T15:06:48.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-15T15:06:48.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-15T15:06:48.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-15T15:06:48.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-15T15:06:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-15T15:06:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-15T15:06:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-15T15:06:48.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-15T15:06:48.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-15T15:06:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-15T15:06:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-15T15:06:48.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-15T15:06:48.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-15T15:06:48.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-15T15:06:48.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-15T15:06:48.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-15T15:06:48.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-15T15:06:48.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-15T15:06:48.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-15T15:06:48.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-15T15:06:48.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-15T15:06:48.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-15T15:06:48.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-15T15:06:48.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-15T15:06:48.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-15T15:06:48.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-15T15:06:48.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-15T15:06:48.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-15T15:06:48.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-15T15:06:48.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-15T15:06:48.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-15T15:06:48.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-15T15:06:48.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-15T15:06:48.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-15T15:06:48.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-15T15:06:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-15T15:06:48.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-15T15:06:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-15T15:06:48.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-15T15:06:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-15T15:06:48.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-15T15:06:48.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-15T15:06:48.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-15T15:06:48.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-15T15:06:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-15T15:06:48.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-15T15:06:48.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-15T15:06:48.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-15T15:06:48.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-15T15:06:48.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-15T15:06:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-15T15:06:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-15T15:06:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-15T15:06:48.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-15T15:06:48.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-15T15:06:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-15T15:06:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-15T15:06:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-15T15:06:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-15T15:06:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-15T15:06:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-15T15:06:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-15T15:06:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-15T15:06:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-15T15:06:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-15T15:06:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-15T15:06:48.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-15T15:06:48.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-15T15:06:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-15T15:06:48.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-15T15:06:48.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-15T15:06:48.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-15T15:06:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-15T15:06:48.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-15T15:06:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-15T15:06:48.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-15T15:06:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-15T15:06:48.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-15T15:06:48.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-15T15:06:48.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-15T15:06:48.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-15T15:06:48.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-15T15:06:48.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-15T15:06:48.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-15T15:06:48.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-15T15:06:48.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-15T15:06:48.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-15T15:06:48.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-15T15:06:48.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-15T15:06:48.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-15T15:06:48.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-15T15:06:48.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-15T15:06:48.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-15T15:06:48.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-15T15:06:48.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-15T15:06:48.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-15T15:06:48.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-15T15:06:48.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-15T15:06:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-15T15:06:48.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-15T15:06:48.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-15T15:06:48.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-15T15:06:48.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-15T15:06:48.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-15T15:06:48.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-15T15:06:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-15T15:06:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-15T15:06:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-15T15:06:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-15T15:06:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-15T15:06:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-15T15:06:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-15T15:06:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-15T15:06:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-15T15:06:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-15T15:06:48.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-15T15:06:48.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-15T15:06:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-15T15:06:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-15T15:06:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-15T15:06:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-15T15:06:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-15T15:06:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-15T15:06:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-15T15:06:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-15T15:06:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-15T15:06:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-15T15:06:48.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-15T15:06:48.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-15T15:06:48.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-15T15:06:48.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-15T15:06:48.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-15T15:06:48.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-15T15:06:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-15T15:06:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-15T15:06:48.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-15T15:06:48.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-15T15:06:48.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-15T15:06:48.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-15T15:06:48.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-15T15:06:48.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-15T15:06:48.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-15T15:06:48.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-15T15:06:48.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-15T15:06:48.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-15T15:06:48.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-15T15:06:48.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-15T15:06:48.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-15T15:06:48.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-15T15:06:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-15T15:06:48.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-15T15:06:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-15T15:06:48.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-15T15:06:48.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-15T15:06:48.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-15T15:06:48.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-15T15:06:48.534 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:06:48.534 DEBUG:teuthology.orchestra.run.smithi107:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-15T15:06:48.558 INFO:teuthology.orchestra.run.smithi107.stdout:postinstall scriptlet (using /bin/sh): 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout:for file in vmlinuz System.map config; do 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: fi 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout:done 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-04-15T15:06:48.559 INFO:teuthology.orchestra.run.smithi107.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-15T15:06:48.560 INFO:teuthology.orchestra.run.smithi107.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:06:48.560 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-04-15T15:06:48.560 INFO:teuthology.orchestra.run.smithi107.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-15T15:06:48.560 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-15T15:06:48.560 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:06:48.560 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-04-15T15:06:48.561 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-04-15T15:06:48.639 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-04-15T15:06:48.639 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-04-15T15:06:48.639 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-04-15T15:06:48.639 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 68.el9 2024-04-15T15:06:48.639 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-04-15T15:06:48.639 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-15T15:06:48.639 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-04-15T15:06:48.639 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8264254 2024-04-15T15:06:48.639 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-04-15T15:06:48.639 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-15T15:06:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-04-15T15:06:48.641 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:06:48.641 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-04-15T15:06:48.641 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-15T15:06:48.642 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:06:48.642 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-15T15:06:49.274 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-04-15T15:06:50.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-04-15T15:06:50.622 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-15T15:06:50.652 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-04-15T15:06:50.657 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-04-15T15:06:50.672 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.ip9Ad9hYn8 2024-04-15T15:06:50.672 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ip9Ad9hYn8 2024-04-15T15:06:50.744 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.ip9Ad9hYn8 2024-04-15T15:06:50.881 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.ip9Ad9hYn8 is 6KB 2024-04-15T15:06:50.930 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.ip9Ad9hYn8 2024-04-15T15:06:50.944 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-04-15T15:06:51.015 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-15T15:06:51.015 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-15T15:06:51.015 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-15T15:06:51.015 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-15T15:06:51.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:06:51.165 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-04-15T15:07:02.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-15T15:07:02.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:07:05.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-04-15T15:07:20.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-15T15:07:20.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:07:21.169 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-15T15:07:21.169 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-15T15:07:21.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:07:23.202 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-04-15T15:07:39.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-04-15T15:07:41.204 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-15T15:07:41.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:07:48.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-15T15:07:48.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:07:51.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-04-15T15:08:03.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-15T15:08:03.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:08:22.175 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-04-15T15:08:37.178 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-15T15:08:37.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:08:40.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-04-15T15:08:41.209 DEBUG:teuthology.orchestra.remote:timed out 2024-04-15T15:08:58.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-15T15:08:58.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:08:58.675 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-04-15T15:08:59.346 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-15T15:08:59.346 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-15T15:08:59.347 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-04-15T15:08:59.388 INFO:teuthology.orchestra.run.smithi107.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:08:59.388 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:08:59.388 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-15T15:08:59.389 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-15T15:09:00.390 DEBUG:teuthology.parallel:result is None 2024-04-15T15:09:02.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-15T15:09:02.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:09:02.689 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-04-15T15:09:03.547 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-15T15:09:03.547 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-15T15:09:03.547 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-04-15T15:09:03.605 INFO:teuthology.orchestra.run.smithi049.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:09:03.605 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:09:03.605 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-15T15:09:03.605 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-15T15:09:04.609 DEBUG:teuthology.parallel:result is None 2024-04-15T15:09:04.609 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-15T15:09:04.617 INFO:teuthology.task.internal:Creating test directory... 2024-04-15T15:09:04.618 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-15T15:09:04.625 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-15T15:09:04.683 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-15T15:09:04.691 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-15T15:09:04.704 INFO:teuthology.task.internal:Creating archive directory... 2024-04-15T15:09:04.704 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-15T15:09:04.710 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-15T15:09:04.784 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-15T15:09:04.791 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-15T15:09:04.792 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-15T15:09:04.800 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-15T15:09:04.940 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:09:04.949 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:09:05.010 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:09:05.021 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:09:05.026 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-15T15:09:05.034 INFO:teuthology.task.internal:Configuring sudo... 2024-04-15T15:09:05.034 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-15T15:09:05.041 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-15T15:09:05.140 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-15T15:09:05.150 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-15T15:09:05.150 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-15T15:09:05.157 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-15T15:09:05.211 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:09:05.295 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:09:05.426 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:09:05.517 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:09:05.626 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:09:05.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-15T15:09:05.729 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:09:05.813 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:09:05.905 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:09:05.989 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:09:06.083 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:09:06.083 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-15T15:09:06.179 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-04-15T15:09:06.186 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-04-15T15:09:06.319 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:09:06.339 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:09:06.810 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-15T15:09:06.820 INFO:teuthology.task.internal:Starting timer... 2024-04-15T15:09:06.821 INFO:teuthology.run_tasks:Running task pcp... 2024-04-15T15:09:06.849 INFO:teuthology.run_tasks:Running task selinux... 2024-04-15T15:09:06.865 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-15T15:09:06.865 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-04-15T15:09:06.998 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-04-15T15:09:07.003 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-04-15T15:09:07.132 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-04-15T15:09:07.137 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-15T15:09:07.137 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-04-15T15:09:07.196 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-04-15T15:09:07.196 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-04-15T15:09:07.249 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-04-15T15:09:07.249 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive'} 2024-04-15T15:09:07.249 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:09:07.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:09:07.339 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:09:07.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:09:07.430 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-15T15:09:07.430 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-04-15T15:09:07.543 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-04-15T15:09:07.647 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-15T15:09:07.658 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-15T15:09:07.675 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-15T15:09:07.676 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi107.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-15T15:15:25.662 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com')] 2024-04-15T15:15:25.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-15T15:15:25.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:15:25.830 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-04-15T15:15:26.000 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-15T15:15:26.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-15T15:15:26.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:15:26.146 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-04-15T15:15:26.316 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-15T15:15:26.316 INFO:teuthology.run_tasks:Running task clock... 2024-04-15T15:15:26.326 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-15T15:15:26.327 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-15T15:15:26.327 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:15:26.335 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-15T15:15:26.335 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:15:26.434 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-15T15:15:26.446 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-15T15:15:26.484 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-15T15:15:26.508 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-15T15:15:26.576 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-04-15T15:15:26.615 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-04-15T15:15:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-04-15T15:15:26.664 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-15T15:15:26.668 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-04-15T15:15:26.711 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-15T15:15:26.727 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-15T15:15:26.787 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-15T15:15:26.840 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:15:26.849 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:15:26.849 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-04-15T15:15:26.849 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:15:26.849 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:15:26.849 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:15:26.849 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:15:26.939 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:15:26.948 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:15:26.949 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-04-15T15:15:26.949 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:15:26.949 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:15:26.949 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:15:26.949 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:15:26.950 INFO:teuthology.run_tasks:Running task install... 2024-04-15T15:15:26.959 DEBUG:teuthology.task.install:project ceph 2024-04-15T15:15:26.959 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}} 2024-04-15T15:15:26.959 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'} 2024-04-15T15:15:26.959 INFO:teuthology.task.install:Using flavor: default 2024-04-15T15:15:26.966 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-15T15:15:26.966 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-15T15:15:26.967 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'tag': None, 'wait_for_package': False} 2024-04-15T15:15:26.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:15:26.970 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'tag': None, 'wait_for_package': False} 2024-04-15T15:15:26.971 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:15:27.119 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/ 2024-04-15T15:15:27.119 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2861.g9bf1c4ba 2024-04-15T15:15:27.185 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/ 2024-04-15T15:15:27.185 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2861.g9bf1c4ba 2024-04-15T15:15:27.228 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-15T15:15:27.229 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:15:27.229 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-15T15:15:27.296 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-15T15:15:27.296 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:15:27.296 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-15T15:15:27.327 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-15T15:15:27.327 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-15T15:15:27.410 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-15T15:15:27.410 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-15T15:15:27.464 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-15T15:15:27.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-15T15:15:27.597 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-15T15:15:27.686 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-04-15T15:15:27.691 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-04-15T15:15:27.714 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-15T15:15:27.819 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-04-15T15:15:27.826 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-04-15T15:15:28.048 INFO:teuthology.orchestra.run.smithi107.stdout:74 files removed 2024-04-15T15:15:28.087 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-15T15:15:28.208 INFO:teuthology.orchestra.run.smithi049.stdout:74 files removed 2024-04-15T15:15:28.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-15T15:15:29.031 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 195 kB/s | 83 kB 00:00 2024-04-15T15:15:29.338 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 153 kB/s | 83 kB 00:00 2024-04-15T15:15:29.456 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-15T15:15:29.913 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-04-15T15:15:29.940 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-15T15:15:30.563 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-04-15T15:15:30.785 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - AppStream 8.7 MB/s | 6.7 MB 00:00 2024-04-15T15:15:31.351 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 9.5 MB/s | 6.7 MB 00:00 2024-04-15T15:15:32.577 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-04-15T15:15:33.018 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-04-15T15:15:35.030 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-04-15T15:15:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-15T15:15:42.105 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 904 kB/s | 8.0 MB 00:09 2024-04-15T15:15:42.806 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-04-15T15:15:44.198 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-15T15:15:45.041 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-04-15T15:15:46.864 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-15T15:15:48.606 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-15T15:15:50.718 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 16 kB 00:00 2024-04-15T15:15:52.754 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-15T15:15:53.050 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-15T15:15:56.150 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:15:56.152 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:15:56.175 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-15T15:15:56.176 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-15T15:15:56.178 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-15T15:15:56.179 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-15T15:15:56.182 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-15T15:15:56.187 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-15T15:15:56.187 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-15T15:15:56.284 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-15T15:15:56.305 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-04-15T15:15:56.305 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-15T15:15:56.305 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-04-15T15:15:56.305 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-15T15:15:56.305 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-15T15:15:56.305 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 6.6 k 2024-04-15T15:15:56.305 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.4 M 2024-04-15T15:15:56.305 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 890 k 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 145 k 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 1.5 M 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 142 k 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 3.5 M 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 7.4 M 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 49 k 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 10 M 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 49 M 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 264 k 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 762 k 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 224 k 2024-04-15T15:15:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 27 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 18 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 32 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 747 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 127 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-15T15:15:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 162 k 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 320 k 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 297 k 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 99 k 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 85 k 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 171 k 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-15T15:15:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.4 M 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:15:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-15T15:15:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-15T15:15:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-15T15:15:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-15T15:15:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-15T15:15:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 21 M 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 34 k 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 2.4 M 2024-04-15T15:15:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 249 k 2024-04-15T15:15:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 4.7 M 2024-04-15T15:15:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 17 M 2024-04-15T15:15:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 16 k 2024-04-15T15:15:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 25 k 2024-04-15T15:15:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-15T15:15:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-15T15:15:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-15T15:15:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-15T15:15:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-15T15:15:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-15T15:15:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-15T15:15:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-15T15:15:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-15T15:15:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 164 k 2024-04-15T15:15:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-15T15:15:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-15T15:15:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-15T15:15:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-15T15:15:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 501 k 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.2 M 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-15T15:15:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-15T15:15:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-15T15:15:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-15T15:15:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-15T15:15:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-15T15:15:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-15T15:15:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-15T15:15:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-15T15:15:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-15T15:15:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-15T15:15:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 45 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 135 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-15T15:15:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-15T15:15:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-15T15:15:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-15T15:15:56.324 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout:Installing weak dependencies: 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-15T15:15:56.325 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-04-15T15:15:56.326 INFO:teuthology.orchestra.run.smithi049.stdout:Install 150 Packages 2024-04-15T15:15:56.326 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 24 Packages 2024-04-15T15:15:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:15:56.326 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 228 M 2024-04-15T15:15:56.326 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-15T15:15:57.685 INFO:teuthology.orchestra.run.smithi049.stdout:(1/174): ceph-19.0.0-2861.g9bf1c4ba.el8.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-15T15:15:58.682 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-15T15:15:59.760 INFO:teuthology.orchestra.run.smithi049.stdout:(2/174): ceph-common-19.0.0-2861.g9bf1c4ba.el8. 8.5 MB/s | 21 MB 00:02 2024-04-15T15:15:59.810 INFO:teuthology.orchestra.run.smithi049.stdout:(3/174): ceph-immutable-object-cache-19.0.0-286 2.9 MB/s | 145 kB 00:00 2024-04-15T15:15:59.977 INFO:teuthology.orchestra.run.smithi049.stdout:(4/174): ceph-mds-19.0.0-2861.g9bf1c4ba.el8.x86 14 MB/s | 2.4 MB 00:00 2024-04-15T15:16:00.044 INFO:teuthology.orchestra.run.smithi049.stdout:(5/174): ceph-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 378 kB/s | 890 kB 00:02 2024-04-15T15:16:00.327 INFO:teuthology.orchestra.run.smithi049.stdout:(6/174): ceph-mon-19.0.0-2861.g9bf1c4ba.el8.x86 17 MB/s | 4.7 MB 00:00 2024-04-15T15:16:01.120 INFO:teuthology.orchestra.run.smithi049.stdout:(7/174): ceph-base-19.0.0-2861.g9bf1c4ba.el8.x8 1.4 MB/s | 5.4 MB 00:03 2024-04-15T15:16:01.345 INFO:teuthology.orchestra.run.smithi049.stdout:(8/174): ceph-osd-19.0.0-2861.g9bf1c4ba.el8.x86 17 MB/s | 17 MB 00:01 2024-04-15T15:16:01.395 INFO:teuthology.orchestra.run.smithi049.stdout:(9/174): ceph-selinux-19.0.0-2861.g9bf1c4ba.el8 518 kB/s | 25 kB 00:00 2024-04-15T15:16:01.625 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:16:01.626 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:16:01.649 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-15T15:16:01.650 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-15T15:16:01.652 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-15T15:16:01.653 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-15T15:16:01.656 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-15T15:16:01.660 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-15T15:16:01.661 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-15T15:16:01.749 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-15T15:16:01.772 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================ 2024-04-15T15:16:01.772 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================ 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 6.6 k 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.4 M 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 890 k 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 145 k 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 1.5 M 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 142 k 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 3.5 M 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 7.4 M 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 49 k 2024-04-15T15:16:01.773 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 10 M 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 49 M 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 264 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 762 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 224 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 27 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 18 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 32 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 747 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 127 k 2024-04-15T15:16:01.774 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 162 k 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 320 k 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 297 k 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 99 k 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 85 k 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 171 k 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-15T15:16:01.775 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-15T15:16:01.776 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.4 M 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-15T15:16:01.777 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 21 M 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 34 k 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 2.4 M 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 249 k 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 4.7 M 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 17 M 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 16 k 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 25 k 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-15T15:16:01.778 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 164 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 501 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.2 M 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-15T15:16:01.779 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-15T15:16:01.780 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-15T15:16:01.780 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-15T15:16:01.780 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-15T15:16:01.780 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-15T15:16:01.782 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-15T15:16:01.782 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-15T15:16:01.782 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-15T15:16:01.782 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-15T15:16:01.782 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 45 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 135 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-15T15:16:01.783 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-15T15:16:01.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-15T15:16:01.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-15T15:16:01.786 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-15T15:16:01.787 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-15T15:16:01.787 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-15T15:16:01.787 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-15T15:16:01.787 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-15T15:16:01.787 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-15T15:16:01.787 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-15T15:16:01.787 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-15T15:16:01.787 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-15T15:16:01.787 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-15T15:16:01.787 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-15T15:16:01.788 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-15T15:16:01.789 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-15T15:16:01.789 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-04-15T15:16:01.789 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-15T15:16:01.789 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:16:01.789 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-15T15:16:01.789 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================ 2024-04-15T15:16:01.789 INFO:teuthology.orchestra.run.smithi107.stdout:Install 150 Packages 2024-04-15T15:16:01.789 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 24 Packages 2024-04-15T15:16:01.789 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:16:01.790 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 228 M 2024-04-15T15:16:01.790 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-15T15:16:02.170 INFO:teuthology.orchestra.run.smithi049.stdout:(10/174): ceph-mgr-19.0.0-2861.g9bf1c4ba.el8.x8 698 kB/s | 1.5 MB 00:02 2024-04-15T15:16:02.496 INFO:teuthology.orchestra.run.smithi049.stdout:(11/174): cephfs-mirror-19.0.0-2861.g9bf1c4ba.e 694 kB/s | 224 kB 00:00 2024-04-15T15:16:02.596 INFO:teuthology.orchestra.run.smithi049.stdout:(12/174): cephfs-shell-19.0.0-2861.g9bf1c4ba.el 273 kB/s | 27 kB 00:00 2024-04-15T15:16:02.696 INFO:teuthology.orchestra.run.smithi049.stdout:(13/174): libcephfs-devel-19.0.0-2861.g9bf1c4ba 320 kB/s | 32 kB 00:00 2024-04-15T15:16:03.146 INFO:teuthology.orchestra.run.smithi049.stdout:(14/174): ceph-radosgw-19.0.0-2861.g9bf1c4ba.el 4.9 MB/s | 10 MB 00:02 2024-04-15T15:16:03.213 INFO:teuthology.orchestra.run.smithi049.stdout:(15/174): libcephsqlite-19.0.0-2861.g9bf1c4ba.e 2.4 MB/s | 164 kB 00:00 2024-04-15T15:16:03.245 INFO:teuthology.orchestra.run.smithi107.stdout:(1/174): ceph-19.0.0-2861.g9bf1c4ba.el8.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-15T15:16:03.288 INFO:teuthology.orchestra.run.smithi049.stdout:(16/174): librados-devel-19.0.0-2861.g9bf1c4ba. 1.7 MB/s | 127 kB 00:00 2024-04-15T15:16:03.489 INFO:teuthology.orchestra.run.smithi049.stdout:(17/174): libradosstriper1-19.0.0-2861.g9bf1c4b 2.5 MB/s | 501 kB 00:00 2024-04-15T15:16:03.926 INFO:teuthology.orchestra.run.smithi107.stdout:(2/174): ceph-base-19.0.0-2861.g9bf1c4ba.el8.x8 4.7 MB/s | 5.4 MB 00:01 2024-04-15T15:16:03.976 INFO:teuthology.orchestra.run.smithi107.stdout:(3/174): ceph-immutable-object-cache-19.0.0-286 2.9 MB/s | 145 kB 00:00 2024-04-15T15:16:04.101 INFO:teuthology.orchestra.run.smithi107.stdout:(4/174): ceph-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 1.0 MB/s | 890 kB 00:00 2024-04-15T15:16:04.112 INFO:teuthology.orchestra.run.smithi049.stdout:(18/174): ceph-test-19.0.0-2861.g9bf1c4ba.el8.x 18 MB/s | 49 MB 00:02 2024-04-15T15:16:04.156 INFO:teuthology.orchestra.run.smithi049.stdout:(19/174): libcephfs2-19.0.0-2861.g9bf1c4ba.el8. 512 kB/s | 747 kB 00:01 2024-04-15T15:16:04.198 INFO:teuthology.orchestra.run.smithi049.stdout:(20/174): python3-ceph-argparse-19.0.0-2861.g9b 536 kB/s | 45 kB 00:00 2024-04-15T15:16:04.298 INFO:teuthology.orchestra.run.smithi049.stdout:(21/174): python3-cephfs-19.0.0-2861.g9bf1c4ba. 1.6 MB/s | 162 kB 00:00 2024-04-15T15:16:04.318 INFO:teuthology.orchestra.run.smithi107.stdout:(5/174): ceph-mds-19.0.0-2861.g9bf1c4ba.el8.x86 6.9 MB/s | 2.4 MB 00:00 2024-04-15T15:16:04.398 INFO:teuthology.orchestra.run.smithi049.stdout:(22/174): python3-rados-19.0.0-2861.g9bf1c4ba.e 3.1 MB/s | 320 kB 00:00 2024-04-15T15:16:04.515 INFO:teuthology.orchestra.run.smithi049.stdout:(23/174): python3-ceph-common-19.0.0-2861.g9bf1 376 kB/s | 135 kB 00:00 2024-04-15T15:16:04.690 INFO:teuthology.orchestra.run.smithi049.stdout:(24/174): librgw2-19.0.0-2861.g9bf1c4ba.el8.x86 4.4 MB/s | 5.2 MB 00:01 2024-04-15T15:16:04.715 INFO:teuthology.orchestra.run.smithi049.stdout:(25/174): python3-rgw-19.0.0-2861.g9bf1c4ba.el8 497 kB/s | 99 kB 00:00 2024-04-15T15:16:04.783 INFO:teuthology.orchestra.run.smithi049.stdout:(26/174): rbd-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 938 kB/s | 85 kB 00:00 2024-04-15T15:16:04.926 INFO:teuthology.orchestra.run.smithi107.stdout:(6/174): ceph-mon-19.0.0-2861.g9bf1c4ba.el8.x86 7.7 MB/s | 4.7 MB 00:00 2024-04-15T15:16:04.933 INFO:teuthology.orchestra.run.smithi049.stdout:(27/174): rbd-nbd-19.0.0-2861.g9bf1c4ba.el8.x86 1.1 MB/s | 171 kB 00:00 2024-04-15T15:16:05.016 INFO:teuthology.orchestra.run.smithi049.stdout:(28/174): python3-rbd-19.0.0-2861.g9bf1c4ba.el8 482 kB/s | 297 kB 00:00 2024-04-15T15:16:05.042 INFO:teuthology.orchestra.run.smithi049.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2861.g 318 kB/s | 34 kB 00:00 2024-04-15T15:16:05.100 INFO:teuthology.orchestra.run.smithi049.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2861.g9bf1c4b 1.7 MB/s | 142 kB 00:00 2024-04-15T15:16:05.627 INFO:teuthology.orchestra.run.smithi107.stdout:(7/174): ceph-mgr-19.0.0-2861.g9bf1c4ba.el8.x86 1.0 MB/s | 1.5 MB 00:01 2024-04-15T15:16:06.526 INFO:teuthology.orchestra.run.smithi049.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-04-15T15:16:06.626 INFO:teuthology.orchestra.run.smithi049.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2861.g9b 2.5 MB/s | 249 kB 00:00 2024-04-15T15:16:06.751 INFO:teuthology.orchestra.run.smithi049.stdout:(33/174): ceph-mgr-rook-19.0.0-2861.g9bf1c4ba.e 398 kB/s | 49 kB 00:00 2024-04-15T15:16:06.842 INFO:teuthology.orchestra.run.smithi049.stdout:(34/174): ceph-mgr-dashboard-19.0.0-2861.g9bf1c 2.0 MB/s | 3.5 MB 00:01 2024-04-15T15:16:06.885 INFO:teuthology.orchestra.run.smithi049.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2861.g9 123 kB/s | 16 kB 00:00 2024-04-15T15:16:07.035 INFO:teuthology.orchestra.run.smithi049.stdout:(36/174): ceph-volume-19.0.0-2861.g9bf1c4ba.el8 1.3 MB/s | 264 kB 00:00 2024-04-15T15:16:07.135 INFO:teuthology.orchestra.run.smithi049.stdout:(37/174): cephadm-19.0.0-2861.g9bf1c4ba.el8.noa 3.0 MB/s | 762 kB 00:00 2024-04-15T15:16:07.194 INFO:teuthology.orchestra.run.smithi049.stdout:(38/174): cephfs-top-19.0.0-2861.g9bf1c4ba.el8. 115 kB/s | 18 kB 00:00 2024-04-15T15:16:07.436 INFO:teuthology.orchestra.run.smithi049.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2024-04-15T15:16:07.478 INFO:teuthology.orchestra.run.smithi049.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 920 kB 00:00 2024-04-15T15:16:07.485 INFO:teuthology.orchestra.run.smithi107.stdout:(8/174): ceph-osd-19.0.0-2861.g9bf1c4ba.el8.x86 6.6 MB/s | 17 MB 00:02 2024-04-15T15:16:07.503 INFO:teuthology.orchestra.run.smithi049.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.7 MB/s | 106 kB 00:00 2024-04-15T15:16:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:(9/174): ceph-selinux-19.0.0-2861.g9bf1c4ba.el8 516 kB/s | 25 kB 00:00 2024-04-15T15:16:07.536 INFO:teuthology.orchestra.run.smithi049.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-04-15T15:16:07.562 INFO:teuthology.orchestra.run.smithi049.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 578 kB/s | 32 kB 00:00 2024-04-15T15:16:07.696 INFO:teuthology.orchestra.run.smithi049.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-15T15:16:07.721 INFO:teuthology.orchestra.run.smithi049.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 116 kB/s | 17 kB 00:00 2024-04-15T15:16:07.763 INFO:teuthology.orchestra.run.smithi049.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.9 MB/s | 159 kB 00:00 2024-04-15T15:16:07.788 INFO:teuthology.orchestra.run.smithi049.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 153 kB/s | 10 kB 00:00 2024-04-15T15:16:07.813 INFO:teuthology.orchestra.run.smithi049.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 888 kB/s | 44 kB 00:00 2024-04-15T15:16:07.863 INFO:teuthology.orchestra.run.smithi049.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-04-15T15:16:07.947 INFO:teuthology.orchestra.run.smithi049.stdout:(50/174): rbd-mirror-19.0.0-2861.g9bf1c4ba.el8. 979 kB/s | 3.1 MB 00:03 2024-04-15T15:16:07.981 INFO:teuthology.orchestra.run.smithi049.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 3.5 MB/s | 578 kB 00:00 2024-04-15T15:16:08.006 INFO:teuthology.orchestra.run.smithi049.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 341 kB/s | 46 kB 00:00 2024-04-15T15:16:08.040 INFO:teuthology.orchestra.run.smithi049.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-04-15T15:16:08.064 INFO:teuthology.orchestra.run.smithi049.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-04-15T15:16:08.098 INFO:teuthology.orchestra.run.smithi049.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.4 MB/s | 217 kB 00:00 2024-04-15T15:16:08.140 INFO:teuthology.orchestra.run.smithi049.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 472 kB/s | 34 kB 00:00 2024-04-15T15:16:08.293 INFO:teuthology.orchestra.run.smithi049.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-15T15:16:08.349 INFO:teuthology.orchestra.run.smithi049.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 6.6 MB/s | 1.5 MB 00:00 2024-04-15T15:16:08.391 INFO:teuthology.orchestra.run.smithi049.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 3.5 MB/s | 817 kB 00:00 2024-04-15T15:16:08.416 INFO:teuthology.orchestra.run.smithi049.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 132 kB 00:00 2024-04-15T15:16:08.483 INFO:teuthology.orchestra.run.smithi049.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-15T15:16:08.796 INFO:teuthology.orchestra.run.smithi049.stdout:(62/174): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.7 MB 00:00 2024-04-15T15:16:08.825 INFO:teuthology.orchestra.run.smithi049.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 199 kB/s | 78 kB 00:00 2024-04-15T15:16:08.984 INFO:teuthology.orchestra.run.smithi049.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 5.8 MB 00:00 2024-04-15T15:16:09.009 INFO:teuthology.orchestra.run.smithi049.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 148 kB/s | 24 kB 00:00 2024-04-15T15:16:09.034 INFO:teuthology.orchestra.run.smithi049.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 95 kB/s | 18 kB 00:00 2024-04-15T15:16:09.110 INFO:teuthology.orchestra.run.smithi049.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-15T15:16:09.134 INFO:teuthology.orchestra.run.smithi049.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 32 kB 00:00 2024-04-15T15:16:09.268 INFO:teuthology.orchestra.run.smithi049.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-04-15T15:16:09.327 INFO:teuthology.orchestra.run.smithi049.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-04-15T15:16:09.343 INFO:teuthology.orchestra.run.smithi049.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 906 kB/s | 174 kB 00:00 2024-04-15T15:16:09.386 INFO:teuthology.orchestra.run.smithi049.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2024-04-15T15:16:09.411 INFO:teuthology.orchestra.run.smithi049.stdout:(73/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-04-15T15:16:09.436 INFO:teuthology.orchestra.run.smithi049.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 238 kB 00:00 2024-04-15T15:16:09.461 INFO:teuthology.orchestra.run.smithi049.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 132 kB 00:00 2024-04-15T15:16:09.486 INFO:teuthology.orchestra.run.smithi049.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.3 MB/s | 211 kB 00:00 2024-04-15T15:16:09.503 INFO:teuthology.orchestra.run.smithi049.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 734 kB/s | 47 kB 00:00 2024-04-15T15:16:09.529 INFO:teuthology.orchestra.run.smithi049.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 670 kB/s | 43 kB 00:00 2024-04-15T15:16:09.604 INFO:teuthology.orchestra.run.smithi049.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 393 kB/s | 39 kB 00:00 2024-04-15T15:16:09.929 INFO:teuthology.orchestra.run.smithi049.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-15T15:16:09.946 INFO:teuthology.orchestra.run.smithi049.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 106 kB/s | 41 kB 00:00 2024-04-15T15:16:09.971 INFO:teuthology.orchestra.run.smithi049.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 623 kB/s | 220 kB 00:00 2024-04-15T15:16:09.988 INFO:teuthology.orchestra.run.smithi049.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 35 kB 00:00 2024-04-15T15:16:10.007 INFO:teuthology.orchestra.run.smithi049.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 46 kB 00:00 2024-04-15T15:16:10.038 INFO:teuthology.orchestra.run.smithi049.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 300 kB 00:00 2024-04-15T15:16:10.064 INFO:teuthology.orchestra.run.smithi049.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-04-15T15:16:10.089 INFO:teuthology.orchestra.run.smithi049.stdout:(87/174): uuid-1.6.2-55.el9.x86_64.rpm 702 kB/s | 56 kB 00:00 2024-04-15T15:16:10.114 INFO:teuthology.orchestra.run.smithi049.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 593 kB/s | 44 kB 00:00 2024-04-15T15:16:10.140 INFO:teuthology.orchestra.run.smithi049.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-15T15:16:10.181 INFO:teuthology.orchestra.run.smithi049.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2024-04-15T15:16:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-04-15T15:16:10.248 INFO:teuthology.orchestra.run.smithi049.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-04-15T15:16:10.273 INFO:teuthology.orchestra.run.smithi049.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.8 MB/s | 187 kB 00:00 2024-04-15T15:16:10.329 INFO:teuthology.orchestra.run.smithi049.stdout:(94/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-04-15T15:16:10.508 INFO:teuthology.orchestra.run.smithi049.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 731 kB/s | 187 kB 00:00 2024-04-15T15:16:10.541 INFO:teuthology.orchestra.run.smithi049.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 925 kB/s | 195 kB 00:00 2024-04-15T15:16:10.566 INFO:teuthology.orchestra.run.smithi049.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-15T15:16:10.602 INFO:teuthology.orchestra.run.smithi107.stdout:(10/174): ceph-common-19.0.0-2861.g9bf1c4ba.el8 2.7 MB/s | 21 MB 00:07 2024-04-15T15:16:10.642 INFO:teuthology.orchestra.run.smithi049.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 217 kB/s | 21 kB 00:00 2024-04-15T15:16:10.684 INFO:teuthology.orchestra.run.smithi049.stdout:(99/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-04-15T15:16:10.709 INFO:teuthology.orchestra.run.smithi049.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 180 kB/s | 25 kB 00:00 2024-04-15T15:16:10.785 INFO:teuthology.orchestra.run.smithi107.stdout:(11/174): cephfs-mirror-19.0.0-2861.g9bf1c4ba.e 1.2 MB/s | 224 kB 00:00 2024-04-15T15:16:10.801 INFO:teuthology.orchestra.run.smithi049.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.1 MB/s | 245 kB 00:00 2024-04-15T15:16:10.860 INFO:teuthology.orchestra.run.smithi049.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-04-15T15:16:10.893 INFO:teuthology.orchestra.run.smithi049.stdout:(103/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.3 MB/s | 248 kB 00:00 2024-04-15T15:16:10.907 INFO:teuthology.orchestra.run.smithi107.stdout:(12/174): cephfs-shell-19.0.0-2861.g9bf1c4ba.el 223 kB/s | 27 kB 00:00 2024-04-15T15:16:10.994 INFO:teuthology.orchestra.run.smithi107.stdout:(13/174): libcephfs-devel-19.0.0-2861.g9bf1c4ba 369 kB/s | 32 kB 00:00 2024-04-15T15:16:11.019 INFO:teuthology.orchestra.run.smithi049.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 11 MB/s | 4.1 MB 00:00 2024-04-15T15:16:11.227 INFO:teuthology.orchestra.run.smithi049.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 61 kB/s | 22 kB 00:00 2024-04-15T15:16:11.319 INFO:teuthology.orchestra.run.smithi107.stdout:(14/174): libcephfs2-19.0.0-2861.g9bf1c4ba.el8. 2.3 MB/s | 747 kB 00:00 2024-04-15T15:16:11.428 INFO:teuthology.orchestra.run.smithi107.stdout:(15/174): libcephsqlite-19.0.0-2861.g9bf1c4ba.e 1.5 MB/s | 164 kB 00:00 2024-04-15T15:16:11.435 INFO:teuthology.orchestra.run.smithi049.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.6 MB/s | 863 kB 00:00 2024-04-15T15:16:11.511 INFO:teuthology.orchestra.run.smithi049.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-15T15:16:11.536 INFO:teuthology.orchestra.run.smithi107.stdout:(16/174): librados-devel-19.0.0-2861.g9bf1c4ba. 1.2 MB/s | 127 kB 00:00 2024-04-15T15:16:11.544 INFO:teuthology.orchestra.run.smithi049.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 986 kB/s | 309 kB 00:00 2024-04-15T15:16:11.569 INFO:teuthology.orchestra.run.smithi049.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 148 kB/s | 19 kB 00:00 2024-04-15T15:16:11.778 INFO:teuthology.orchestra.run.smithi049.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-15T15:16:11.786 INFO:teuthology.orchestra.run.smithi107.stdout:(17/174): libradosstriper1-19.0.0-2861.g9bf1c4b 2.0 MB/s | 501 kB 00:00 2024-04-15T15:16:11.803 INFO:teuthology.orchestra.run.smithi049.stdout:(111/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 116 kB/s | 26 kB 00:00 2024-04-15T15:16:11.862 INFO:teuthology.orchestra.run.smithi049.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 172 kB/s | 49 kB 00:00 2024-04-15T15:16:11.896 INFO:teuthology.orchestra.run.smithi049.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 584 kB/s | 67 kB 00:00 2024-04-15T15:16:11.938 INFO:teuthology.orchestra.run.smithi049.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-15T15:16:11.979 INFO:teuthology.orchestra.run.smithi049.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.1 MB/s | 838 kB 00:00 2024-04-15T15:16:12.021 INFO:teuthology.orchestra.run.smithi049.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-15T15:16:12.046 INFO:teuthology.orchestra.run.smithi049.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 423 kB/s | 43 kB 00:00 2024-04-15T15:16:12.072 INFO:teuthology.orchestra.run.smithi049.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 377 kB/s | 32 kB 00:00 2024-04-15T15:16:12.114 INFO:teuthology.orchestra.run.smithi049.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 230 kB/s | 15 kB 00:00 2024-04-15T15:16:12.139 INFO:teuthology.orchestra.run.smithi049.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.9 MB/s | 172 kB 00:00 2024-04-15T15:16:12.172 INFO:teuthology.orchestra.run.smithi049.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-15T15:16:12.231 INFO:teuthology.orchestra.run.smithi049.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-15T15:16:12.256 INFO:teuthology.orchestra.run.smithi049.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 428 kB/s | 35 kB 00:00 2024-04-15T15:16:12.281 INFO:teuthology.orchestra.run.smithi049.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-04-15T15:16:12.362 INFO:teuthology.orchestra.run.smithi107.stdout:(18/174): ceph-radosgw-19.0.0-2861.g9bf1c4ba.el 1.5 MB/s | 10 MB 00:06 2024-04-15T15:16:12.398 INFO:teuthology.orchestra.run.smithi049.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-15T15:16:12.423 INFO:teuthology.orchestra.run.smithi049.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-15T15:16:12.449 INFO:teuthology.orchestra.run.smithi049.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2024-04-15T15:16:12.516 INFO:teuthology.orchestra.run.smithi049.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 271 kB/s | 18 kB 00:00 2024-04-15T15:16:12.541 INFO:teuthology.orchestra.run.smithi049.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 256 kB/s | 23 kB 00:00 2024-04-15T15:16:12.566 INFO:teuthology.orchestra.run.smithi049.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 169 kB/s | 19 kB 00:00 2024-04-15T15:16:12.591 INFO:teuthology.orchestra.run.smithi049.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 265 kB/s | 20 kB 00:00 2024-04-15T15:16:12.658 INFO:teuthology.orchestra.run.smithi049.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-15T15:16:12.683 INFO:teuthology.orchestra.run.smithi049.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 399 kB/s | 46 kB 00:00 2024-04-15T15:16:12.709 INFO:teuthology.orchestra.run.smithi049.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 677 kB/s | 79 kB 00:00 2024-04-15T15:16:12.759 INFO:teuthology.orchestra.run.smithi049.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 788 kB/s | 58 kB 00:00 2024-04-15T15:16:12.784 INFO:teuthology.orchestra.run.smithi049.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-15T15:16:12.809 INFO:teuthology.orchestra.run.smithi049.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-15T15:16:12.883 INFO:teuthology.orchestra.run.smithi049.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 921 kB/s | 90 kB 00:00 2024-04-15T15:16:12.910 INFO:teuthology.orchestra.run.smithi049.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 228 kB/s | 28 kB 00:00 2024-04-15T15:16:12.912 INFO:teuthology.orchestra.run.smithi107.stdout:(19/174): ceph-test-19.0.0-2861.g9bf1c4ba.el8.x 9.0 MB/s | 49 MB 00:05 2024-04-15T15:16:12.935 INFO:teuthology.orchestra.run.smithi049.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2024-04-15T15:16:12.937 INFO:teuthology.orchestra.run.smithi107.stdout:(20/174): python3-ceph-argparse-19.0.0-2861.g9b 78 kB/s | 45 kB 00:00 2024-04-15T15:16:12.971 INFO:teuthology.orchestra.run.smithi107.stdout:(21/174): python3-ceph-common-19.0.0-2861.g9bf1 2.3 MB/s | 135 kB 00:00 2024-04-15T15:16:12.977 INFO:teuthology.orchestra.run.smithi049.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 902 kB/s | 59 kB 00:00 2024-04-15T15:16:13.003 INFO:teuthology.orchestra.run.smithi049.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-15T15:16:13.028 INFO:teuthology.orchestra.run.smithi049.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2024-04-15T15:16:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 692 kB/s | 51 kB 00:00 2024-04-15T15:16:13.063 INFO:teuthology.orchestra.run.smithi107.stdout:(22/174): python3-rados-19.0.0-2861.g9bf1c4ba.e 3.4 MB/s | 320 kB 00:00 2024-04-15T15:16:13.087 INFO:teuthology.orchestra.run.smithi049.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-15T15:16:13.112 INFO:teuthology.orchestra.run.smithi049.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-15T15:16:13.138 INFO:teuthology.orchestra.run.smithi107.stdout:(23/174): python3-rbd-19.0.0-2861.g9bf1c4ba.el8 3.9 MB/s | 297 kB 00:00 2024-04-15T15:16:13.145 INFO:teuthology.orchestra.run.smithi049.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.7 MB/s | 427 kB 00:00 2024-04-15T15:16:13.164 INFO:teuthology.orchestra.run.smithi107.stdout:(24/174): python3-cephfs-19.0.0-2861.g9bf1c4ba. 720 kB/s | 162 kB 00:00 2024-04-15T15:16:13.171 INFO:teuthology.orchestra.run.smithi049.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 243 kB/s | 20 kB 00:00 2024-04-15T15:16:13.197 INFO:teuthology.orchestra.run.smithi107.stdout:(25/174): python3-rgw-19.0.0-2861.g9bf1c4ba.el8 1.7 MB/s | 99 kB 00:00 2024-04-15T15:16:13.204 INFO:teuthology.orchestra.run.smithi049.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-04-15T15:16:13.231 INFO:teuthology.orchestra.run.smithi107.stdout:(26/174): rbd-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 1.3 MB/s | 85 kB 00:00 2024-04-15T15:16:13.329 INFO:teuthology.orchestra.run.smithi049.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-15T15:16:13.398 INFO:teuthology.orchestra.run.smithi107.stdout:(27/174): librgw2-19.0.0-2861.g9bf1c4ba.el8.x86 3.2 MB/s | 5.2 MB 00:01 2024-04-15T15:16:13.423 INFO:teuthology.orchestra.run.smithi107.stdout:(28/174): rbd-nbd-19.0.0-2861.g9bf1c4ba.el8.x86 894 kB/s | 171 kB 00:00 2024-04-15T15:16:13.457 INFO:teuthology.orchestra.run.smithi107.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2861.g 593 kB/s | 34 kB 00:00 2024-04-15T15:16:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:(30/174): rbd-mirror-19.0.0-2861.g9bf1c4ba.el8. 8.6 MB/s | 3.1 MB 00:00 2024-04-15T15:16:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:(31/174): ceph-mgr-cephadm-19.0.0-2861.g9bf1c4b 899 kB/s | 142 kB 00:00 2024-04-15T15:16:13.613 INFO:teuthology.orchestra.run.smithi049.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-04-15T15:16:13.649 INFO:teuthology.orchestra.run.smithi107.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2861.g9b 3.7 MB/s | 249 kB 00:00 2024-04-15T15:16:13.697 INFO:teuthology.orchestra.run.smithi049.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-04-15T15:16:13.723 INFO:teuthology.orchestra.run.smithi049.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.7 MB/s | 41 kB 00:00 2024-04-15T15:16:13.767 INFO:teuthology.orchestra.run.smithi107.stdout:(33/174): ceph-mgr-rook-19.0.0-2861.g9bf1c4ba.e 421 kB/s | 49 kB 00:00 2024-04-15T15:16:13.816 INFO:teuthology.orchestra.run.smithi107.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2861.g9 341 kB/s | 16 kB 00:00 2024-04-15T15:16:13.875 INFO:teuthology.orchestra.run.smithi107.stdout:(35/174): ceph-volume-19.0.0-2861.g9bf1c4ba.el8 4.5 MB/s | 264 kB 00:00 2024-04-15T15:16:13.947 INFO:teuthology.orchestra.run.smithi049.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 1.1 MB/s | 248 kB 00:00 2024-04-15T15:16:13.975 INFO:teuthology.orchestra.run.smithi107.stdout:(36/174): cephadm-19.0.0-2861.g9bf1c4ba.el8.noa 7.5 MB/s | 762 kB 00:00 2024-04-15T15:16:14.025 INFO:teuthology.orchestra.run.smithi107.stdout:(37/174): cephfs-top-19.0.0-2861.g9bf1c4ba.el8. 370 kB/s | 18 kB 00:00 2024-04-15T15:16:14.050 INFO:teuthology.orchestra.run.smithi049.stdout:(155/174): librados2-19.0.0-2861.g9bf1c4ba.el8. 3.9 MB/s | 3.4 MB 00:00 2024-04-15T15:16:14.084 INFO:teuthology.orchestra.run.smithi049.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.3 MB/s | 321 kB 00:00 2024-04-15T15:16:14.134 INFO:teuthology.orchestra.run.smithi049.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 894 kB/s | 71 kB 00:00 2024-04-15T15:16:14.217 INFO:teuthology.orchestra.run.smithi107.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-04-15T15:16:14.218 INFO:teuthology.orchestra.run.smithi049.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 274 kB/s | 36 kB 00:00 2024-04-15T15:16:14.252 INFO:teuthology.orchestra.run.smithi049.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-04-15T15:16:14.276 INFO:teuthology.orchestra.run.smithi107.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 920 kB 00:00 2024-04-15T15:16:14.310 INFO:teuthology.orchestra.run.smithi049.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 402 kB/s | 23 kB 00:00 2024-04-15T15:16:14.360 INFO:teuthology.orchestra.run.smithi107.stdout:(40/174): ceph-mgr-dashboard-19.0.0-2861.g9bf1c 3.9 MB/s | 3.5 MB 00:00 2024-04-15T15:16:14.369 INFO:teuthology.orchestra.run.smithi049.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.7 MB/s | 256 kB 00:00 2024-04-15T15:16:14.384 INFO:teuthology.orchestra.run.smithi107.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 982 kB/s | 106 kB 00:00 2024-04-15T15:16:14.394 INFO:teuthology.orchestra.run.smithi049.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-04-15T15:16:14.418 INFO:teuthology.orchestra.run.smithi107.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-04-15T15:16:14.444 INFO:teuthology.orchestra.run.smithi107.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 574 kB/s | 32 kB 00:00 2024-04-15T15:16:14.461 INFO:teuthology.orchestra.run.smithi049.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.7 MB/s | 155 kB 00:00 2024-04-15T15:16:14.486 INFO:teuthology.orchestra.run.smithi049.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 136 kB 00:00 2024-04-15T15:16:14.520 INFO:teuthology.orchestra.run.smithi049.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-04-15T15:16:14.544 INFO:teuthology.orchestra.run.smithi107.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-15T15:16:14.545 INFO:teuthology.orchestra.run.smithi049.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 497 kB/s | 29 kB 00:00 2024-04-15T15:16:14.569 INFO:teuthology.orchestra.run.smithi107.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 155 kB/s | 17 kB 00:00 2024-04-15T15:16:14.595 INFO:teuthology.orchestra.run.smithi107.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 205 kB/s | 10 kB 00:00 2024-04-15T15:16:14.620 INFO:teuthology.orchestra.run.smithi107.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-15T15:16:14.637 INFO:teuthology.orchestra.run.smithi049.stdout:(167/174): librbd1-19.0.0-2861.g9bf1c4ba.el8.x8 2.3 MB/s | 3.1 MB 00:01 2024-04-15T15:16:14.645 INFO:teuthology.orchestra.run.smithi107.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 920 kB/s | 44 kB 00:00 2024-04-15T15:16:14.695 INFO:teuthology.orchestra.run.smithi107.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-04-15T15:16:14.762 INFO:teuthology.orchestra.run.smithi107.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 5.1 MB/s | 578 kB 00:00 2024-04-15T15:16:14.787 INFO:teuthology.orchestra.run.smithi107.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 539 kB/s | 46 kB 00:00 2024-04-15T15:16:14.821 INFO:teuthology.orchestra.run.smithi107.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-04-15T15:16:14.854 INFO:teuthology.orchestra.run.smithi049.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.6 MB/s | 1.2 MB 00:00 2024-04-15T15:16:14.863 INFO:teuthology.orchestra.run.smithi107.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2024-04-15T15:16:14.889 INFO:teuthology.orchestra.run.smithi107.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-04-15T15:16:14.914 INFO:teuthology.orchestra.run.smithi107.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 721 kB/s | 34 kB 00:00 2024-04-15T15:16:15.054 INFO:teuthology.orchestra.run.smithi049.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 2.3 MB/s | 465 kB 00:00 2024-04-15T15:16:15.104 INFO:teuthology.orchestra.run.smithi049.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 635 kB/s | 31 kB 00:00 2024-04-15T15:16:15.148 INFO:teuthology.orchestra.run.smithi107.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-15T15:16:15.198 INFO:teuthology.orchestra.run.smithi107.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 5.7 MB/s | 1.5 MB 00:00 2024-04-15T15:16:15.213 INFO:teuthology.orchestra.run.smithi049.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.2 MB/s | 2.2 MB 00:00 2024-04-15T15:16:15.248 INFO:teuthology.orchestra.run.smithi107.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 8.1 MB/s | 817 kB 00:00 2024-04-15T15:16:15.265 INFO:teuthology.orchestra.run.smithi107.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 132 kB 00:00 2024-04-15T15:16:15.305 INFO:teuthology.orchestra.run.smithi049.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 821 kB/s | 74 kB 00:00 2024-04-15T15:16:15.389 INFO:teuthology.orchestra.run.smithi049.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.8 MB/s | 1.1 MB 00:00 2024-04-15T15:16:15.507 INFO:teuthology.orchestra.run.smithi107.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.7 MB 00:00 2024-04-15T15:16:15.549 INFO:teuthology.orchestra.run.smithi107.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-04-15T15:16:15.572 INFO:teuthology.orchestra.run.smithi049.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.4 MB/s | 2.3 MB 00:00 2024-04-15T15:16:15.582 INFO:teuthology.orchestra.run.smithi107.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 2.4 MB/s | 78 kB 00:00 2024-04-15T15:16:15.587 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:16:15.588 INFO:teuthology.orchestra.run.smithi049.stdout:Total 12 MB/s | 228 MB 00:19 2024-04-15T15:16:15.758 INFO:teuthology.orchestra.run.smithi107.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-04-15T15:16:15.774 INFO:teuthology.orchestra.run.smithi107.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 138 kB/s | 24 kB 00:00 2024-04-15T15:16:15.816 INFO:teuthology.orchestra.run.smithi107.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 447 kB/s | 18 kB 00:00 2024-04-15T15:16:15.842 INFO:teuthology.orchestra.run.smithi107.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 144 kB 00:00 2024-04-15T15:16:15.868 INFO:teuthology.orchestra.run.smithi107.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 653 kB/s | 32 kB 00:00 2024-04-15T15:16:15.992 INFO:teuthology.orchestra.run.smithi107.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-04-15T15:16:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-04-15T15:16:16.060 INFO:teuthology.orchestra.run.smithi107.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-04-15T15:16:16.076 INFO:teuthology.orchestra.run.smithi107.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 69 kB 00:00 2024-04-15T15:16:16.102 INFO:teuthology.orchestra.run.smithi107.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2024-04-15T15:16:16.135 INFO:teuthology.orchestra.run.smithi107.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-04-15T15:16:16.161 INFO:teuthology.orchestra.run.smithi107.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-04-15T15:16:16.194 INFO:teuthology.orchestra.run.smithi107.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-04-15T15:16:16.220 INFO:teuthology.orchestra.run.smithi107.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 847 kB/s | 47 kB 00:00 2024-04-15T15:16:16.245 INFO:teuthology.orchestra.run.smithi107.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 873 kB/s | 43 kB 00:00 2024-04-15T15:16:16.470 INFO:teuthology.orchestra.run.smithi107.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 177 kB/s | 39 kB 00:00 2024-04-15T15:16:16.696 INFO:teuthology.orchestra.run.smithi107.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-04-15T15:16:16.721 INFO:teuthology.orchestra.run.smithi107.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 165 kB/s | 41 kB 00:00 2024-04-15T15:16:16.753 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-15T15:16:16.754 INFO:teuthology.orchestra.run.smithi107.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.8 MB/s | 220 kB 00:00 2024-04-15T15:16:16.790 INFO:teuthology.orchestra.run.smithi107.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 35 kB 00:00 2024-04-15T15:16:16.813 INFO:teuthology.orchestra.run.smithi107.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 803 kB/s | 46 kB 00:00 2024-04-15T15:16:16.847 INFO:teuthology.orchestra.run.smithi107.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 300 kB 00:00 2024-04-15T15:16:16.872 INFO:teuthology.orchestra.run.smithi107.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-04-15T15:16:16.898 INFO:teuthology.orchestra.run.smithi107.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-04-15T15:16:16.923 INFO:teuthology.orchestra.run.smithi107.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 902 kB/s | 44 kB 00:00 2024-04-15T15:16:16.948 INFO:teuthology.orchestra.run.smithi107.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-04-15T15:16:16.982 INFO:teuthology.orchestra.run.smithi107.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-04-15T15:16:17.040 INFO:teuthology.orchestra.run.smithi107.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-04-15T15:16:17.066 INFO:teuthology.orchestra.run.smithi107.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-04-15T15:16:17.091 INFO:teuthology.orchestra.run.smithi107.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-04-15T15:16:17.125 INFO:teuthology.orchestra.run.smithi107.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-04-15T15:16:17.159 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-15T15:16:17.159 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-15T15:16:17.267 INFO:teuthology.orchestra.run.smithi107.stdout:(94/174): ceph-mgr-diskprediction-local-19.0.0- 2.0 MB/s | 7.4 MB 00:03 2024-04-15T15:16:17.384 INFO:teuthology.orchestra.run.smithi107.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 646 kB/s | 187 kB 00:00 2024-04-15T15:16:17.417 INFO:teuthology.orchestra.run.smithi107.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-15T15:16:17.451 INFO:teuthology.orchestra.run.smithi107.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-15T15:16:17.476 INFO:teuthology.orchestra.run.smithi107.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 371 kB/s | 21 kB 00:00 2024-04-15T15:16:17.493 INFO:teuthology.orchestra.run.smithi107.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 619 kB/s | 25 kB 00:00 2024-04-15T15:16:17.534 INFO:teuthology.orchestra.run.smithi107.stdout:(100/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-04-15T15:16:17.568 INFO:teuthology.orchestra.run.smithi107.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 3.2 MB/s | 245 kB 00:00 2024-04-15T15:16:17.602 INFO:teuthology.orchestra.run.smithi107.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 3.6 MB/s | 248 kB 00:00 2024-04-15T15:16:17.644 INFO:teuthology.orchestra.run.smithi107.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-15T15:16:17.686 INFO:teuthology.orchestra.run.smithi107.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 270 kB/s | 22 kB 00:00 2024-04-15T15:16:17.786 INFO:teuthology.orchestra.run.smithi107.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 13 MB/s | 4.1 MB 00:00 2024-04-15T15:16:17.904 INFO:teuthology.orchestra.run.smithi107.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-04-15T15:16:18.254 INFO:teuthology.orchestra.run.smithi107.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 67 kB/s | 19 kB 00:00 2024-04-15T15:16:18.287 INFO:teuthology.orchestra.run.smithi107.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 914 kB/s | 548 kB 00:00 2024-04-15T15:16:18.321 INFO:teuthology.orchestra.run.smithi107.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 658 kB/s | 309 kB 00:00 2024-04-15T15:16:18.363 INFO:teuthology.orchestra.run.smithi107.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 349 kB/s | 26 kB 00:00 2024-04-15T15:16:18.405 INFO:teuthology.orchestra.run.smithi107.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 585 kB/s | 49 kB 00:00 2024-04-15T15:16:18.438 INFO:teuthology.orchestra.run.smithi107.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 902 kB/s | 67 kB 00:00 2024-04-15T15:16:18.497 INFO:teuthology.orchestra.run.smithi107.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-15T15:16:18.597 INFO:teuthology.orchestra.run.smithi107.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2024-04-15T15:16:18.656 INFO:teuthology.orchestra.run.smithi107.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-15T15:16:18.681 INFO:teuthology.orchestra.run.smithi107.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 527 kB/s | 43 kB 00:00 2024-04-15T15:16:18.723 INFO:teuthology.orchestra.run.smithi107.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 488 kB/s | 32 kB 00:00 2024-04-15T15:16:18.748 INFO:teuthology.orchestra.run.smithi107.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 226 kB/s | 15 kB 00:00 2024-04-15T15:16:19.007 INFO:teuthology.orchestra.run.smithi107.stdout:(119/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-04-15T15:16:19.041 INFO:teuthology.orchestra.run.smithi107.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 542 kB/s | 172 kB 00:00 2024-04-15T15:16:19.074 INFO:teuthology.orchestra.run.smithi107.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2024-04-15T15:16:19.108 INFO:teuthology.orchestra.run.smithi107.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-15T15:16:19.133 INFO:teuthology.orchestra.run.smithi107.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 386 kB/s | 35 kB 00:00 2024-04-15T15:16:19.158 INFO:teuthology.orchestra.run.smithi107.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-15T15:16:19.224 INFO:teuthology.orchestra.run.smithi107.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-04-15T15:16:19.258 INFO:teuthology.orchestra.run.smithi107.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 108 kB/s | 11 kB 00:00 2024-04-15T15:16:19.316 INFO:teuthology.orchestra.run.smithi107.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-04-15T15:16:19.342 INFO:teuthology.orchestra.run.smithi107.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-04-15T15:16:19.367 INFO:teuthology.orchestra.run.smithi107.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-15T15:16:19.393 INFO:teuthology.orchestra.run.smithi107.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-04-15T15:16:19.443 INFO:teuthology.orchestra.run.smithi107.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 264 kB/s | 20 kB 00:00 2024-04-15T15:16:19.468 INFO:teuthology.orchestra.run.smithi107.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 620 kB/s | 46 kB 00:00 2024-04-15T15:16:19.551 INFO:teuthology.orchestra.run.smithi107.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-04-15T15:16:19.585 INFO:teuthology.orchestra.run.smithi107.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 496 kB/s | 58 kB 00:00 2024-04-15T15:16:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:(135/174): python3-more-itertools-8.12.0-2.el9. 473 kB/s | 79 kB 00:00 2024-04-15T15:16:19.644 INFO:teuthology.orchestra.run.smithi107.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-15T15:16:19.669 INFO:teuthology.orchestra.run.smithi107.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-04-15T15:16:19.694 INFO:teuthology.orchestra.run.smithi107.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-15T15:16:19.720 INFO:teuthology.orchestra.run.smithi107.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 378 kB/s | 28 kB 00:00 2024-04-15T15:16:19.745 INFO:teuthology.orchestra.run.smithi107.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 412 kB/s | 31 kB 00:00 2024-04-15T15:16:19.766 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-15T15:16:19.767 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-15T15:16:19.770 INFO:teuthology.orchestra.run.smithi107.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-15T15:16:19.795 INFO:teuthology.orchestra.run.smithi107.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 791 kB/s | 59 kB 00:00 2024-04-15T15:16:19.821 INFO:teuthology.orchestra.run.smithi107.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-04-15T15:16:19.846 INFO:teuthology.orchestra.run.smithi107.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2024-04-15T15:16:19.880 INFO:teuthology.orchestra.run.smithi107.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-15T15:16:19.905 INFO:teuthology.orchestra.run.smithi107.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-15T15:16:19.938 INFO:teuthology.orchestra.run.smithi107.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-04-15T15:16:19.964 INFO:teuthology.orchestra.run.smithi107.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-04-15T15:16:19.997 INFO:teuthology.orchestra.run.smithi107.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-15T15:16:20.073 INFO:teuthology.orchestra.run.smithi107.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-15T15:16:20.173 INFO:teuthology.orchestra.run.smithi107.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 23 MB/s | 2.3 MB 00:00 2024-04-15T15:16:20.266 INFO:teuthology.orchestra.run.smithi107.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-04-15T15:16:20.298 INFO:teuthology.orchestra.run.smithi107.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-15T15:16:20.340 INFO:teuthology.orchestra.run.smithi107.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 6.0 MB/s | 248 kB 00:00 2024-04-15T15:16:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-04-15T15:16:20.466 INFO:teuthology.orchestra.run.smithi107.stdout:(156/174): librbd1-19.0.0-2861.g9bf1c4ba.el8.x8 6.7 MB/s | 3.1 MB 00:00 2024-04-15T15:16:20.521 INFO:teuthology.orchestra.run.smithi107.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 520 kB/s | 71 kB 00:00 2024-04-15T15:16:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 439 kB/s | 36 kB 00:00 2024-04-15T15:16:20.584 INFO:teuthology.orchestra.run.smithi107.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.0 MB/s | 192 kB 00:00 2024-04-15T15:16:20.626 INFO:teuthology.orchestra.run.smithi107.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 564 kB/s | 23 kB 00:00 2024-04-15T15:16:20.659 INFO:teuthology.orchestra.run.smithi107.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-04-15T15:16:20.701 INFO:teuthology.orchestra.run.smithi107.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-04-15T15:16:20.735 INFO:teuthology.orchestra.run.smithi107.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-04-15T15:16:20.760 INFO:teuthology.orchestra.run.smithi107.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-04-15T15:16:20.785 INFO:teuthology.orchestra.run.smithi107.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.3 MB/s | 64 kB 00:00 2024-04-15T15:16:20.810 INFO:teuthology.orchestra.run.smithi107.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 578 kB/s | 29 kB 00:00 2024-04-15T15:16:20.886 INFO:teuthology.orchestra.run.smithi107.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-04-15T15:16:20.986 INFO:teuthology.orchestra.run.smithi107.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2024-04-15T15:16:21.061 INFO:teuthology.orchestra.run.smithi107.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 13 MB/s | 2.3 MB 00:00 2024-04-15T15:16:21.103 INFO:teuthology.orchestra.run.smithi107.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.9 MB/s | 465 kB 00:00 2024-04-15T15:16:21.128 INFO:teuthology.orchestra.run.smithi107.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 471 kB/s | 31 kB 00:00 2024-04-15T15:16:21.196 INFO:teuthology.orchestra.run.smithi107.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-04-15T15:16:21.212 INFO:teuthology.orchestra.run.smithi107.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 899 kB/s | 74 kB 00:00 2024-04-15T15:16:21.620 INFO:teuthology.orchestra.run.smithi107.stdout:(174/174): librados2-19.0.0-2861.g9bf1c4ba.el8. 2.1 MB/s | 3.4 MB 00:01 2024-04-15T15:16:21.635 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:16:21.635 INFO:teuthology.orchestra.run.smithi107.stdout:Total 11 MB/s | 228 MB 00:19 2024-04-15T15:16:22.819 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-15T15:16:23.197 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-15T15:16:23.197 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-15T15:16:25.558 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-15T15:16:25.559 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-15T15:16:27.386 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-15T15:16:27.495 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-15T15:16:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-15T15:16:27.814 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:16:27.860 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:16:27.963 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-15T15:16:28.065 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-15T15:16:28.118 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-15T15:16:28.147 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-15T15:16:28.239 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-15T15:16:28.667 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-15T15:16:28.712 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:16:28.906 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:16:28.939 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:16:29.066 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:16:29.104 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:16:29.168 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-15T15:16:29.466 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-15T15:16:29.518 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-15T15:16:29.535 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:16:29.694 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:16:29.719 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:16:29.814 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:16:29.872 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-15T15:16:29.896 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:16:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:16:30.260 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:16:30.294 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:16:30.359 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-15T15:16:30.694 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-15T15:16:30.878 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:16:30.968 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:16:31.183 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:16:31.239 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:16:31.581 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:16:31.777 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-15T15:16:31.902 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-15T15:16:31.946 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-15T15:16:31.983 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-15T15:16:32.045 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-15T15:16:32.167 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-15T15:16:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-15T15:16:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-15T15:16:32.392 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-15T15:16:32.435 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-15T15:16:32.496 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-15T15:16:32.537 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-15T15:16:32.571 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-15T15:16:32.687 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-15T15:16:32.726 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 39/198 2024-04-15T15:16:32.777 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 40/198 2024-04-15T15:16:32.824 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 41/198 2024-04-15T15:16:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-15T15:16:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-15T15:16:32.949 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-15T15:16:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-15T15:16:33.010 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-15T15:16:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:16:33.091 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:16:33.114 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-15T15:16:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-15T15:16:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-15T15:16:33.296 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-15T15:16:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-15T15:16:33.364 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-15T15:16:33.378 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-15T15:16:33.465 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-15T15:16:33.884 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-15T15:16:33.956 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:16:34.121 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:16:34.126 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-15T15:16:34.144 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:16:34.223 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-15T15:16:34.265 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:16:34.292 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:16:34.334 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-15T15:16:34.361 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-15T15:16:34.406 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:16:34.435 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:16:34.490 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-15T15:16:34.564 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-15T15:16:34.623 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 53/198 2024-04-15T15:16:34.653 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-15T15:16:34.669 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-15T15:16:34.715 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-15T15:16:34.740 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:16:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:16:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:16:35.009 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:16:35.051 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-15T15:16:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:16:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-15T15:16:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:16:35.306 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-15T15:16:35.392 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-15T15:16:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:16:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:16:35.478 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-15T15:16:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-15T15:16:35.695 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-15T15:16:35.754 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-15T15:16:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-15T15:16:35.830 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-15T15:16:35.905 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-15T15:16:35.962 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-15T15:16:36.002 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-15T15:16:36.016 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:16:36.071 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-15T15:16:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:16:36.119 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-15T15:16:36.184 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-15T15:16:36.253 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-15T15:16:36.305 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:16:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:16:36.470 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-15T15:16:36.533 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-15T15:16:36.572 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-15T15:16:36.631 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-15T15:16:36.714 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:16:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-15T15:16:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-15T15:16:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-15T15:16:36.897 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-15T15:16:36.902 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-15T15:16:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-15T15:16:37.020 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-15T15:16:37.058 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-15T15:16:37.095 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-15T15:16:37.136 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-15T15:16:37.138 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-15T15:16:37.240 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-15T15:16:37.306 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-15T15:16:37.315 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-15T15:16:37.348 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-15T15:16:37.436 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-15T15:16:37.472 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-15T15:16:37.480 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-15T15:16:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-15T15:16:37.539 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-15T15:16:37.576 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-15T15:16:37.592 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-15T15:16:37.608 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-15T15:16:37.667 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-15T15:16:37.718 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-15T15:16:37.753 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 39/198 2024-04-15T15:16:37.847 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 40/198 2024-04-15T15:16:37.878 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 41/198 2024-04-15T15:16:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-15T15:16:37.984 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-15T15:16:37.996 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-15T15:16:38.054 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-15T15:16:38.154 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-15T15:16:38.163 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-15T15:16:38.214 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-15T15:16:38.254 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-15T15:16:38.292 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:16:38.334 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-15T15:16:38.335 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-15T15:16:38.372 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-15T15:16:38.400 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-15T15:16:38.430 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-15T15:16:38.487 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-15T15:16:38.525 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-15T15:16:38.670 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-15T15:16:38.708 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-15T15:16:38.743 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-15T15:16:38.806 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-15T15:16:38.847 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-15T15:16:38.995 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-15T15:16:39.083 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-15T15:16:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-15T15:16:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-15T15:16:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-15T15:16:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-15T15:16:39.308 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-15T15:16:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-15T15:16:39.355 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:16:39.383 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:16:39.436 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-15T15:16:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-15T15:16:39.495 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-15T15:16:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-15T15:16:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 53/198 2024-04-15T15:16:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-15T15:16:40.000 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-15T15:16:40.188 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-15T15:16:40.268 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-15T15:16:40.326 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-15T15:16:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-15T15:16:40.533 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-15T15:16:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-15T15:16:40.580 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-15T15:16:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-15T15:16:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-15T15:16:40.668 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-15T15:16:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-15T15:16:40.772 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-15T15:16:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-15T15:16:40.811 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-15T15:16:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-15T15:16:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-15T15:16:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-15T15:16:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-15T15:16:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-15T15:16:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-15T15:16:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-15T15:16:41.172 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-15T15:16:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-15T15:16:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-15T15:16:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-15T15:16:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-15T15:16:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-15T15:16:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-15T15:16:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-15T15:16:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-15T15:16:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-15T15:16:41.926 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-15T15:16:42.037 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-15T15:16:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-15T15:16:42.168 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-15T15:16:42.241 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-15T15:16:42.291 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-15T15:16:42.354 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-15T15:16:42.427 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-15T15:16:42.681 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-15T15:16:42.752 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-15T15:16:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-15T15:16:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-15T15:16:43.014 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-15T15:16:43.052 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:16:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-15T15:16:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-15T15:16:43.197 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-15T15:16:43.235 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-15T15:16:43.279 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-15T15:16:43.402 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-15T15:16:43.443 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-15T15:16:43.478 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-15T15:16:43.532 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-15T15:16:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-15T15:16:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-15T15:16:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-15T15:16:43.844 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-15T15:16:43.904 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-15T15:16:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-15T15:16:43.940 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-15T15:16:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-15T15:16:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-15T15:16:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-15T15:16:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-15T15:16:44.134 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-15T15:16:44.197 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-15T15:16:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-15T15:16:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:16:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:16:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 125/198 2024-04-15T15:16:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-15T15:16:45.151 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-15T15:16:45.205 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-15T15:16:45.238 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-15T15:16:45.253 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-15T15:16:45.281 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-15T15:16:45.378 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-15T15:16:45.414 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-15T15:16:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:16:45.602 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:16:45.633 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 129/198 2024-04-15T15:16:45.673 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-15T15:16:45.703 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-15T15:16:45.797 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-15T15:16:46.595 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-15T15:16:46.724 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-15T15:16:48.284 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-15T15:16:48.392 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-15T15:16:48.429 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-15T15:16:48.434 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:16:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-15T15:16:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:16:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-15T15:16:48.884 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-15T15:16:48.916 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:16:49.036 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:16:49.064 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 125/198 2024-04-15T15:16:49.120 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-15T15:16:49.167 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:16:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:16:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:16:49.391 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-15T15:16:49.391 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-15T15:16:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:16:49.432 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:16:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-15T15:16:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:16:49.768 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:16:49.796 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 129/198 2024-04-15T15:16:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:16:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:16:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:16:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:16:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:16:53.184 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-15T15:16:53.184 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-15T15:16:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:16:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:17:02.863 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:17:02.863 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-04-15T15:17:02.863 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-04-15T15:17:02.863 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-04-15T15:17:02.864 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-04-15T15:17:02.864 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-04-15T15:17:02.864 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-04-15T15:17:02.864 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-04-15T15:17:02.864 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:03.096 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:17:03.142 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:17:03.143 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:03.143 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-15T15:17:03.143 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:04.815 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:17:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:17:04.890 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:04.890 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-15T15:17:04.890 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:17:04.890 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:17:04.890 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:05.183 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:17:05.258 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:17:05.258 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:05.258 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-15T15:17:05.258 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:17:05.258 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:17:05.259 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:05.809 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:17:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:17:05.889 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:05.889 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-15T15:17:05.889 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:17:05.889 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:17:05.889 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:05.981 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:17:05.981 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-04-15T15:17:05.981 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-04-15T15:17:05.981 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-04-15T15:17:05.981 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-04-15T15:17:05.981 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-04-15T15:17:05.981 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-04-15T15:17:05.981 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-04-15T15:17:05.981 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:06.019 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-15T15:17:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:17:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:17:06.250 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:06.250 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-15T15:17:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:06.681 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:17:06.720 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:17:06.772 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 139/198 2024-04-15T15:17:06.851 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 140/198 2024-04-15T15:17:06.882 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:17:07.842 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:17:07.948 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:17:07.948 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:07.948 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-15T15:17:07.948 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:17:07.948 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:17:07.948 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:08.192 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:17:08.218 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:17:08.258 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:17:08.287 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:17:08.287 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:08.288 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-15T15:17:08.288 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:17:08.288 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:17:08.288 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:08.776 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:17:08.871 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:17:08.871 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:08.871 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-15T15:17:08.871 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:17:08.871 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:17:08.872 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:08.987 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-15T15:17:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:17:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:17:09.496 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:17:09.563 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:17:09.600 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:17:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 139/198 2024-04-15T15:17:09.731 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 140/198 2024-04-15T15:17:09.758 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:17:09.781 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 144/198 2024-04-15T15:17:09.810 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:17:09.885 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:17:09.885 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:09.885 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-15T15:17:09.885 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:17:09.885 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:17:09.886 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:09.953 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:17:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:17:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 147/198 2024-04-15T15:17:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:17:11.075 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-15T15:17:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:17:11.113 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:17:11.188 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:17:11.188 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:11.188 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-15T15:17:11.188 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:17:11.188 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:17:11.188 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:11.264 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:17:11.367 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:17:11.367 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:11.367 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-15T15:17:11.367 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:17:11.562 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:17:11.563 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:11.563 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-15T15:17:11.563 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:17:11.563 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:17:11.563 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:11.974 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:17:12.081 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:17:12.081 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:12.081 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-15T15:17:12.081 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:17:12.081 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:17:12.081 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:17:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:17:12.382 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:17:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 144/198 2024-04-15T15:17:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:17:12.690 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:17:12.690 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:12.690 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-15T15:17:12.691 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:17:12.691 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:17:12.691 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:12.747 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:17:12.784 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:17:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 147/198 2024-04-15T15:17:13.800 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-15T15:17:13.832 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:17:13.899 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:17:13.900 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:13.900 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-15T15:17:13.900 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:17:13.900 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:17:13.900 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:13.950 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:17:14.020 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:17:14.020 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:14.020 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-15T15:17:14.021 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:14.109 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:17:14.185 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:17:14.186 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:14.186 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-15T15:17:14.186 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:17:14.186 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:17:14.186 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:14.576 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:17:14.649 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:17:14.649 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:17:14.649 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-15T15:17:14.649 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:17:14.649 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:17:14.649 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:17.334 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:17:17.392 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 154/198 2024-04-15T15:17:17.430 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 155/198 2024-04-15T15:17:17.470 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-15T15:17:17.793 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 157/198 2024-04-15T15:17:17.974 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-15T15:17:18.025 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 159/198 2024-04-15T15:17:18.087 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-15T15:17:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-15T15:17:18.218 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-15T15:17:18.284 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 163/198 2024-04-15T15:17:18.363 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-15T15:17:18.416 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-15T15:17:18.512 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-15T15:17:18.639 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-15T15:17:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-15T15:17:19.559 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:17:19.601 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:17:19.601 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:17:19.657 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 154/198 2024-04-15T15:17:19.666 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:17:19.698 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 155/198 2024-04-15T15:17:19.737 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-15T15:17:20.020 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 157/198 2024-04-15T15:17:20.142 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-15T15:17:20.199 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 159/198 2024-04-15T15:17:20.255 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-15T15:17:20.316 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-15T15:17:20.361 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-15T15:17:20.404 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 163/198 2024-04-15T15:17:20.440 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-15T15:17:20.492 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-15T15:17:20.578 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-15T15:17:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-15T15:17:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-15T15:17:21.111 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 170/198 2024-04-15T15:17:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-15T15:17:21.272 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-15T15:17:21.416 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-15T15:17:21.459 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-15T15:17:21.492 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-15T15:17:21.527 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-15T15:17:21.570 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-15T15:17:21.571 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:17:21.724 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:17:21.733 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:17:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:17:21.773 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-15T15:17:21.773 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:17:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:17:21.952 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:17:22.001 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-15T15:17:22.050 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-15T15:17:22.083 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-15T15:17:22.116 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-15T15:17:22.189 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-15T15:17:22.267 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-15T15:17:22.315 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-15T15:17:22.350 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-15T15:17:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-15T15:17:22.440 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:17:22.473 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-15T15:17:22.511 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:17:22.512 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:17:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:17:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:17:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:17:23.106 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:17:23.143 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-15T15:17:23.158 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 170/198 2024-04-15T15:17:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-15T15:17:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-15T15:17:23.253 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-15T15:17:23.304 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-15T15:17:23.354 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:17:23.436 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-15T15:17:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-15T15:17:23.505 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-15T15:17:23.540 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-15T15:17:23.591 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-15T15:17:23.591 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:17:23.634 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:17:23.634 INFO:teuthology.orchestra.run.smithi049.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-15T15:17:23.634 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:23.744 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:17:23.786 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-15T15:17:23.786 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:17:23.938 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:17:23.993 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-15T15:17:24.041 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-15T15:17:24.075 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-15T15:17:24.107 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-15T15:17:24.183 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-15T15:17:24.273 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-15T15:17:24.321 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-15T15:17:24.354 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-15T15:17:24.388 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-15T15:17:24.421 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:17:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-15T15:17:24.491 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:17:24.491 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:17:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:17:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:17:24.994 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:17:25.046 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:17:25.083 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-15T15:17:25.119 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-15T15:17:25.159 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-15T15:17:25.279 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:17:25.534 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:17:25.535 INFO:teuthology.orchestra.run.smithi107.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-15T15:17:25.535 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:28.985 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 1/198 2024-04-15T15:17:28.985 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2/198 2024-04-15T15:17:28.985 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 3/198 2024-04-15T15:17:28.985 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 4/198 2024-04-15T15:17:28.985 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 5/198 2024-04-15T15:17:28.985 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 6/198 2024-04-15T15:17:28.985 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 7/198 2024-04-15T15:17:28.985 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 8/198 2024-04-15T15:17:28.985 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 9/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 13/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 14/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 15/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 17/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 18/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 19/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 20/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 21/198 2024-04-15T15:17:28.986 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 22/198 2024-04-15T15:17:28.987 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 23/198 2024-04-15T15:17:28.987 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 24/198 2024-04-15T15:17:28.987 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 25/198 2024-04-15T15:17:29.003 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 26/198 2024-04-15T15:17:29.003 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 27/198 2024-04-15T15:17:29.003 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 28/198 2024-04-15T15:17:29.003 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 29/198 2024-04-15T15:17:29.003 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 30/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 31/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 32/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 33/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 34/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 35/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 36/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 37/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 38/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 39/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-15T15:17:29.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-15T15:17:29.005 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-15T15:17:29.006 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-15T15:17:29.007 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-15T15:17:29.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-15T15:17:29.009 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-15T15:17:29.010 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-15T15:17:29.011 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:17:29.012 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-15T15:17:29.013 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-15T15:17:29.014 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-15T15:17:29.015 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:17:29.016 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-15T15:17:29.016 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-15T15:17:29.016 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-15T15:17:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 1/198 2024-04-15T15:17:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2/198 2024-04-15T15:17:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 3/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 4/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 5/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 6/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 7/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 8/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 9/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 13/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 14/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 15/198 2024-04-15T15:17:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 17/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 18/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 19/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 20/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 21/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 22/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 23/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 24/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 25/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 26/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 27/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 28/198 2024-04-15T15:17:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 29/198 2024-04-15T15:17:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 30/198 2024-04-15T15:17:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 31/198 2024-04-15T15:17:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 32/198 2024-04-15T15:17:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 33/198 2024-04-15T15:17:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 34/198 2024-04-15T15:17:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 35/198 2024-04-15T15:17:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 36/198 2024-04-15T15:17:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 37/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 38/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 39/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-15T15:17:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-15T15:17:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-15T15:17:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-15T15:17:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-15T15:17:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-15T15:17:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-15T15:17:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-15T15:17:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-15T15:17:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-15T15:17:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-15T15:17:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-15T15:17:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-15T15:17:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-15T15:17:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-15T15:17:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-15T15:17:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-15T15:17:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-15T15:17:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-15T15:17:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-15T15:17:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-15T15:17:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:17:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-15T15:17:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-15T15:17:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-15T15:17:31.634 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-15T15:17:31.634 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:31.634 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-04-15T15:17:31.634 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-15T15:17:31.634 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-15T15:17:31.634 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-15T15:17:31.634 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-15T15:17:31.634 INFO:teuthology.orchestra.run.smithi049.stdout: acl-2.3.1-4.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-15T15:17:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.2-8.el9.noarch 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.7.4-5.el9.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.636 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.637 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.4-9.el9.x86_64 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-15T15:17:31.638 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: jq-1.6-16.el9.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-15T15:17:31.639 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: lua-5.4.4-4.el9.x86_64 2024-04-15T15:17:31.640 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-13.14-1.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-15T15:17:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-15T15:17:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-15T15:17:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-04-15T15:17:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-15T15:17:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-15T15:17:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-3.el9.x86_64 2024-04-15T15:17:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-15T15:17:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-15T15:17:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-15T15:17:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-15T15:17:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-15T15:17:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-15T15:17:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:31.648 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-15T15:17:32.501 DEBUG:teuthology.parallel:result is None 2024-04-15T15:17:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-15T15:17:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: acl-2.3.1-4.el9.x86_64 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-15T15:17:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-15T15:17:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-15T15:17:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: jq-1.6-16.el9.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-15T15:17:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-15T15:17:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-15T15:17:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-15T15:17:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-15T15:17:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-15T15:17:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-04-15T15:17:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-15T15:17:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-15T15:17:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.14-1.el9.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-15T15:17:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-15T15:17:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-15T15:17:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-04-15T15:17:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-15T15:17:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-3.el9.x86_64 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-15T15:17:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-15T15:17:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-15T15:17:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-15T15:17:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-15T15:17:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-15T15:17:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:33.334 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-15T15:17:34.069 DEBUG:teuthology.parallel:result is None 2024-04-15T15:17:34.070 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:17:34.309 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-15T15:17:34.361 INFO:teuthology.orchestra.run.smithi049.stdout:19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:17:34.363 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:17:34.363 INFO:teuthology.task.install:The correct ceph version 19.0.0-2861.g9bf1c4ba is installed. 2024-04-15T15:17:34.364 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:17:34.505 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-15T15:17:34.565 INFO:teuthology.orchestra.run.smithi107.stdout:19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:17:34.566 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:17:34.566 INFO:teuthology.task.install:The correct ceph version 19.0.0-2861.g9bf1c4ba is installed. 2024-04-15T15:17:34.568 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-15T15:17:34.568 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:34.568 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-15T15:17:34.648 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:17:34.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-15T15:17:34.725 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-15T15:17:34.725 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:34.725 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-04-15T15:17:34.796 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-15T15:17:34.918 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:17:34.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-04-15T15:17:34.993 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-15T15:17:35.106 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-15T15:17:35.106 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:35.106 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-15T15:17:35.189 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-15T15:17:35.310 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:17:35.310 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-15T15:17:35.385 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-15T15:17:35.498 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-15T15:17:35.499 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:35.499 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-04-15T15:17:35.587 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-15T15:17:35.742 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:17:35.742 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-04-15T15:17:35.825 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-15T15:17:35.937 INFO:teuthology.run_tasks:Running task ceph... 2024-04-15T15:17:36.034 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-15T15:17:36.035 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /var/log/ceph 2024-04-15T15:17:36.040 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /var/log/ceph 2024-04-15T15:17:36.117 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-15T15:17:36.117 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-15T15:17:36.121 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-15T15:17:36.193 INFO:tasks.ceph:Creating extra log directories... 2024-04-15T15:17:36.194 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-15T15:17:36.198 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-15T15:17:36.283 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-15T15:17:36.284 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-15T15:17:36.284 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-15T15:17:36.284 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:36.284 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-15T15:17:36.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-15T15:17:36.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-15T15:17:36.669 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:17:36.669 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-15T15:17:36.711 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-15T15:17:36.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-15T15:17:36.998 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-15T15:17:36.998 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-15T15:17:36.998 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119', 'branch': 'wip-lusov-quiesce-xlock', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/quiesce}', 'email': None, 'first_in_suite': False, 'job_id': '7657119', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-quiesce-xlock'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-quiesce-xlock', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2288, 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-lusov-quiesce-xlock', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi049.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDB2/YdZYsSxPRvJ82wegIo13jQ1h3vGBQsXIodUm9Dh1Wb3P75Jv3mchNNDhv2hcIMgigAsLdPTiTAdgMChKRA=', 'smithi107.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJQqSxuRx5wTNkPdbKM8TVcVuTeoPB/CT9UuQMYOsso+CKgHzs0p5W4GWzB0DfADwhDMn8308WtVk0DNgnQRUI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_quiesce']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-15_14:46:31', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-15T15:17:36.998 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-15T15:17:37.006 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-15T15:17:37.060 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/run/ceph 2024-04-15T15:17:37.065 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/run/ceph 2024-04-15T15:17:37.161 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:37.161 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-04-15T15:17:37.218 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-15T15:17:37.219 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-04-15T15:17:37.272 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-15T15:17:37.272 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:17:37.272 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-15T15:17:37.273 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:17:37.273 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:17:37.273 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-15 15:17:24.108923455 +0000 2024-04-15T15:17:37.273 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-15 15:14:52.180289660 +0000 2024-04-15T15:17:37.273 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-15 15:14:52.180289660 +0000 2024-04-15T15:17:37.273 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-04-15 15:14:52.180289660 +0000 2024-04-15T15:17:37.273 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-15T15:17:37.370 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-15T15:17:37.371 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-15T15:17:37.371 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.00027815 s, 1.8 MB/s 2024-04-15T15:17:37.375 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-15T15:17:37.432 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-04-15T15:17:37.484 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-15T15:17:37.484 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:17:37.484 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-15T15:17:37.484 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:17:37.485 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:17:37.485 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-15 15:17:24.109923428 +0000 2024-04-15T15:17:37.485 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-15 15:14:52.784273092 +0000 2024-04-15T15:17:37.485 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-15 15:14:52.785273064 +0000 2024-04-15T15:17:37.485 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-04-15 15:14:52.784273092 +0000 2024-04-15T15:17:37.486 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-15T15:17:37.581 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-15T15:17:37.582 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-15T15:17:37.582 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000273248 s, 1.9 MB/s 2024-04-15T15:17:37.587 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-15T15:17:37.647 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-04-15T15:17:37.698 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-15T15:17:37.698 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:17:37.698 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-15T15:17:37.699 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:17:37.699 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:17:37.699 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-15 15:17:24.109923428 +0000 2024-04-15T15:17:37.699 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-15 15:14:53.421255618 +0000 2024-04-15T15:17:37.699 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-15 15:14:53.421255618 +0000 2024-04-15T15:17:37.699 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-04-15 15:14:53.421255618 +0000 2024-04-15T15:17:37.699 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-15T15:17:37.793 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-15T15:17:37.794 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-15T15:17:37.794 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000250067 s, 2.0 MB/s 2024-04-15T15:17:37.798 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-15T15:17:37.855 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-04-15T15:17:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-15T15:17:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:17:37.907 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-15T15:17:37.907 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:17:37.907 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:17:37.907 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-15 15:17:24.110923401 +0000 2024-04-15T15:17:37.907 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-15 15:14:54.069237842 +0000 2024-04-15T15:17:37.907 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-15 15:14:54.069237842 +0000 2024-04-15T15:17:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-04-15 15:14:54.069237842 +0000 2024-04-15T15:17:37.907 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-15T15:17:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-15T15:17:38.001 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-15T15:17:38.001 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000251772 s, 2.0 MB/s 2024-04-15T15:17:38.004 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-15T15:17:38.062 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:17:38.063 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:17:38.063 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-04-15T15:17:38.121 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-15T15:17:38.121 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-04-15T15:17:38.170 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-15T15:17:38.170 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:17:38.170 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-15T15:17:38.170 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:17:38.170 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:17:38.170 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-15 15:17:26.016385330 +0000 2024-04-15T15:17:38.170 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-15 15:14:40.928769726 +0000 2024-04-15T15:17:38.170 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-15 15:14:40.928769726 +0000 2024-04-15T15:17:38.171 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-15 15:14:40.928769726 +0000 2024-04-15T15:17:38.171 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-15T15:17:38.265 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-15T15:17:38.266 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-15T15:17:38.266 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000364859 s, 1.4 MB/s 2024-04-15T15:17:38.269 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-15T15:17:38.321 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-04-15T15:17:38.367 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-15T15:17:38.367 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:17:38.367 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-15T15:17:38.367 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:17:38.367 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:17:38.367 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-15 15:17:26.017385313 +0000 2024-04-15T15:17:38.367 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-15 15:14:41.737756501 +0000 2024-04-15T15:17:38.367 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-15 15:14:41.737756501 +0000 2024-04-15T15:17:38.367 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-15 15:14:41.737756501 +0000 2024-04-15T15:17:38.368 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-15T15:17:38.504 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-15T15:17:38.504 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-15T15:17:38.504 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000308381 s, 1.7 MB/s 2024-04-15T15:17:38.508 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-15T15:17:38.564 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-04-15T15:17:38.613 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-15T15:17:38.613 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:17:38.613 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-15T15:17:38.613 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:17:38.613 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:17:38.613 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-15 15:17:26.017385313 +0000 2024-04-15T15:17:38.613 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-15 15:14:42.358746349 +0000 2024-04-15T15:17:38.613 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-15 15:14:42.358746349 +0000 2024-04-15T15:17:38.613 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-15 15:14:42.358746349 +0000 2024-04-15T15:17:38.614 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-15T15:17:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-15T15:17:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-15T15:17:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000285078 s, 1.8 MB/s 2024-04-15T15:17:38.694 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-15T15:17:38.747 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-04-15T15:17:38.797 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-15T15:17:38.797 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:17:38.798 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-15T15:17:38.798 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:17:38.798 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:17:38.798 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-15 15:17:26.018385297 +0000 2024-04-15T15:17:38.798 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-15 15:14:42.962736476 +0000 2024-04-15T15:17:38.798 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-15 15:14:42.962736476 +0000 2024-04-15T15:17:38.798 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-15 15:14:42.962736476 +0000 2024-04-15T15:17:38.798 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-15T15:17:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-15T15:17:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-15T15:17:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000462187 s, 1.1 MB/s 2024-04-15T15:17:38.884 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-15T15:17:38.937 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:17:38.937 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-15T15:17:38.937 INFO:tasks.ceph:Generating config... 2024-04-15T15:17:38.938 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-15T15:17:38.938 INFO:tasks.ceph:[client] debug client = 20 2024-04-15T15:17:38.938 INFO:tasks.ceph:[client] debug ms = 1 2024-04-15T15:17:38.939 INFO:tasks.ceph:[client] pid file = 2024-04-15T15:17:38.939 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-15T15:17:38.939 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-15T15:17:38.939 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-15T15:17:38.939 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-15T15:17:38.939 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-15T15:17:38.939 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-15T15:17:38.939 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-15T15:17:38.939 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-15T15:17:38.939 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-15T15:17:38.939 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-15T15:17:38.939 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-15T15:17:38.939 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-15T15:17:38.939 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-15T15:17:38.940 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-15T15:17:38.940 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-15T15:17:38.940 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-15T15:17:38.940 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-15T15:17:38.940 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-15T15:17:38.940 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-15T15:17:38.940 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-15T15:17:38.940 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-15T15:17:38.940 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-15T15:17:38.940 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-15T15:17:38.940 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-15T15:17:38.940 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-15T15:17:38.940 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-15T15:17:38.940 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-15T15:17:38.940 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-15T15:17:38.940 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-15T15:17:38.941 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-15T15:17:38.941 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-15T15:17:38.941 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-15T15:17:38.941 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-15T15:17:38.941 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-15T15:17:38.941 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-15T15:17:38.941 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-15T15:17:38.941 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-15T15:17:38.941 INFO:tasks.ceph:Setting up mon.a... 2024-04-15T15:17:38.941 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-15T15:17:39.108 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.keyring 2024-04-15T15:17:39.118 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-15T15:17:39.266 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-15T15:17:39.376 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.49'), ('mon.b', '172.21.15.107'), ('mon.c', '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]')] 2024-04-15T15:17:39.377 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.49,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-15T15:17:39.378 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:39.378 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-15T15:17:39.438 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.49 --add b 172.21.15.107 --addv c '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: generated fsid 9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c 2024-04-15T15:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = quincy 2024-04-15T15:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 0 2024-04-15T15:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:fsid 9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c 2024-04-15T15:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:last_changed 2024-04-15T15:17:39.600159+0000 2024-04-15T15:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-04-15T15:17:39.600159+0000 2024-04-15T15:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:min_mon_release 17 (quincy) 2024-04-15T15:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:election_strategy: 1 2024-04-15T15:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-04-15T15:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-04-15T15:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:2: [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] mon.c 2024-04-15T15:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-15T15:17:39.604 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-15T15:17:39.659 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c... 2024-04-15T15:17:39.660 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-15T15:17:39.665 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-15T15:17:39.836 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-04-15T15:17:39.836 INFO:teuthology.orchestra.run.smithi107.stdout: chdir = "" 2024-04-15T15:17:39.836 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-15T15:17:39.836 INFO:teuthology.orchestra.run.smithi107.stdout: auth supported = cephx 2024-04-15T15:17:39.836 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.836 INFO:teuthology.orchestra.run.smithi107.stdout: filestore xattr use omap = true 2024-04-15T15:17:39.836 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.836 INFO:teuthology.orchestra.run.smithi107.stdout: mon clock drift allowed = 1.000 2024-04-15T15:17:39.836 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: osd crush chooseleaf type = 0 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: auth debug = true 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on old message = true 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on bug = true 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: mon pg warn max object skew = 0 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: # disable pg_autoscaler by default for new pools 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default size = 2 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow primary affinity = true 2024-04-15T15:17:39.837 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow pg remap = true 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on legacy crush tunables = false 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on crush straw calc version zero = false 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on no sortbitwise = false 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on osd down out interval zero = false 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on too few osds = false 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: mon_allow_pool_size_one = true 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: osd default data pool replay window = 5 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: mon allow pool delete = true 2024-04-15T15:17:39.838 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: mon cluster log file level = debug 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: debug asserts on shutdown = true 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: mon health detail to clog = false 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: mon host = "172.21.15.49,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]" 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on skipped message = False 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = 9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout:[osd] 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: osd journal size = 100 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub load threshold = 5.0 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub max interval = 600 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: osd mclock profile = high_recovery_ops 2024-04-15T15:17:39.839 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: osd recover clone overlap = true 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: osd recovery max chunk = 1048576 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug shutdown = true 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug op order = true 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify stray on activate = true 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug trim objects = true 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: osd open classes on start = true 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug pg log writeout = true 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: osd deep scrub update digest min age = 30 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.840 INFO:teuthology.orchestra.run.smithi107.stdout: osd map max advance = 10 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: journal zero on create = true 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: filestore ondisk finisher threads = 3 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: filestore apply finisher threads = 3 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: bdev debug aio = true 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug misdirected ops = true 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: bdev async discard = True 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: bdev enable discard = True 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore allocator = bitmap 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore block size = 96636764160 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore fsck on mount = True 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluefs = 1/20 2024-04-15T15:17:39.841 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluestore = 1/20 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: debug osd = 20 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: debug rocksdb = 4/10 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd backfillfull_ratio = 0.85 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd full ratio = 0.9 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd nearfull ratio = 0.8 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: osd failsafe full ratio = 0.95 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: osd objectstore = bluestore 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr] 2024-04-15T15:17:39.842 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-04-15T15:17:39.843 INFO:teuthology.orchestra.run.smithi107.stdout: debug mgr = 20 2024-04-15T15:17:39.844 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2024-04-15T15:17:39.844 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2024-04-15T15:17:39.844 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:17:39.844 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:17:39.844 INFO:teuthology.orchestra.run.smithi107.stdout: mgr/telemetry/nag = false 2024-04-15T15:17:39.844 INFO:teuthology.orchestra.run.smithi107.stdout: debug client = 20 2024-04-15T15:17:39.845 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.845 INFO:teuthology.orchestra.run.smithi107.stdout:[mon] 2024-04-15T15:17:39.845 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-04-15T15:17:39.845 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2024-04-15T15:17:39.845 INFO:teuthology.orchestra.run.smithi107.stdout: debug paxos = 20 2024-04-15T15:17:39.845 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2024-04-15T15:17:39.846 INFO:teuthology.orchestra.run.smithi107.stdout: mon data avail warn = 5 2024-04-15T15:17:39.846 INFO:teuthology.orchestra.run.smithi107.stdout: mon mgr mkfs grace = 240 2024-04-15T15:17:39.846 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:17:39.846 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd reporter subtree level = osd 2024-04-15T15:17:39.846 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd prime pg temp = true 2024-04-15T15:17:39.846 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:17:39.847 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.847 INFO:teuthology.orchestra.run.smithi107.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-15T15:17:39.847 INFO:teuthology.orchestra.run.smithi107.stdout: auth mon ticket ttl = 660 # 11m 2024-04-15T15:17:39.847 INFO:teuthology.orchestra.run.smithi107.stdout: auth service ticket ttl = 240 # 4m 2024-04-15T15:17:39.847 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.847 INFO:teuthology.orchestra.run.smithi107.stdout: # don't complain about insecure global_id in the test suite 2024-04-15T15:17:39.848 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-15T15:17:39.848 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-15T15:17:39.848 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.848 INFO:teuthology.orchestra.run.smithi107.stdout: # 1m isn't quite enough 2024-04-15T15:17:39.848 INFO:teuthology.orchestra.run.smithi107.stdout: mon_down_mkfs_grace = 2m 2024-04-15T15:17:39.848 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.849 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_filestore_osds = false 2024-04-15T15:17:39.849 INFO:teuthology.orchestra.run.smithi107.stdout: mon down mkfs grace = 300 2024-04-15T15:17:39.849 INFO:teuthology.orchestra.run.smithi107.stdout: mon op complaint time = 120 2024-04-15T15:17:39.849 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-15T15:17:39.849 INFO:teuthology.orchestra.run.smithi107.stdout:[client] 2024-04-15T15:17:39.849 INFO:teuthology.orchestra.run.smithi107.stdout: rgw cache enabled = true 2024-04-15T15:17:39.850 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable ops log = true 2024-04-15T15:17:39.850 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable usage log = true 2024-04-15T15:17:39.850 INFO:teuthology.orchestra.run.smithi107.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-15T15:17:39.850 INFO:teuthology.orchestra.run.smithi107.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-15T15:17:39.850 INFO:teuthology.orchestra.run.smithi107.stdout: client mount timeout = 600 2024-04-15T15:17:39.850 INFO:teuthology.orchestra.run.smithi107.stdout: debug client = 20 2024-04-15T15:17:39.851 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-04-15T15:17:39.851 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = "" 2024-04-15T15:17:39.851 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 900 2024-04-15T15:17:39.851 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 900 2024-04-15T15:17:39.851 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.a] 2024-04-15T15:17:39.851 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.b] 2024-04-15T15:17:39.852 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.c] 2024-04-15T15:17:39.852 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2024-04-15T15:17:39.852 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c] 2024-04-15T15:17:39.852 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2024-04-15T15:17:39.852 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d] 2024-04-15T15:17:39.852 INFO:teuthology.orchestra.run.smithi107.stdout:[mds] 2024-04-15T15:17:39.852 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds = 20 2024-04-15T15:17:39.853 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds balancer = 20 2024-04-15T15:17:39.853 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-04-15T15:17:39.853 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug frag = True 2024-04-15T15:17:39.853 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug scatterstat = True 2024-04-15T15:17:39.853 INFO:teuthology.orchestra.run.smithi107.stdout: mds op complaint time = 180 2024-04-15T15:17:39.853 INFO:teuthology.orchestra.run.smithi107.stdout: mds verify scatter = True 2024-04-15T15:17:39.854 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2024-04-15T15:17:39.854 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 900 2024-04-15T15:17:39.854 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 900 2024-04-15T15:17:39.861 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-04-15T15:17:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2024-04-15T15:17:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2024-04-15T15:17:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-15T15:17:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-15T15:17:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2024-04-15T15:17:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-15T15:17:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2024-04-15T15:17:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2024-04-15T15:17:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2024-04-15T15:17:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2024-04-15T15:17:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2024-04-15T15:17:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.49,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]" 2024-04-15T15:17:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on skipped message = False 2024-04-15T15:17:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c 2024-04-15T15:17:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.866 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2024-04-15T15:17:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2024-04-15T15:17:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2024-04-15T15:17:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2024-04-15T15:17:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2024-04-15T15:17:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2024-04-15T15:17:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2024-04-15T15:17:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2024-04-15T15:17:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2024-04-15T15:17:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2024-04-15T15:17:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug trim objects = true 2024-04-15T15:17:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2024-04-15T15:17:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2024-04-15T15:17:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2024-04-15T15:17:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2024-04-15T15:17:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2024-04-15T15:17:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2024-04-15T15:17:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2024-04-15T15:17:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2024-04-15T15:17:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2024-04-15T15:17:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: bdev async discard = True 2024-04-15T15:17:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: bdev enable discard = True 2024-04-15T15:17:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore allocator = bitmap 2024-04-15T15:17:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2024-04-15T15:17:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2024-04-15T15:17:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2024-04-15T15:17:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2024-04-15T15:17:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-04-15T15:17:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2024-04-15T15:17:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2024-04-15T15:17:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2024-04-15T15:17:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2024-04-15T15:17:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2024-04-15T15:17:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2024-04-15T15:17:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2024-04-15T15:17:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2024-04-15T15:17:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.874 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2024-04-15T15:17:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-04-15T15:17:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2024-04-15T15:17:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2024-04-15T15:17:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2024-04-15T15:17:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:17:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:17:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2024-04-15T15:17:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2024-04-15T15:17:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.875 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2024-04-15T15:17:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-04-15T15:17:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2024-04-15T15:17:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2024-04-15T15:17:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2024-04-15T15:17:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2024-04-15T15:17:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2024-04-15T15:17:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:17:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2024-04-15T15:17:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2024-04-15T15:17:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:17:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-15T15:17:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2024-04-15T15:17:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2024-04-15T15:17:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2024-04-15T15:17:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-15T15:17:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-15T15:17:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2024-04-15T15:17:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2024-04-15T15:17:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2024-04-15T15:17:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: mon down mkfs grace = 300 2024-04-15T15:17:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: mon op complaint time = 120 2024-04-15T15:17:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:17:39.879 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2024-04-15T15:17:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2024-04-15T15:17:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2024-04-15T15:17:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2024-04-15T15:17:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-15T15:17:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-15T15:17:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: client mount timeout = 600 2024-04-15T15:17:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2024-04-15T15:17:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-04-15T15:17:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = "" 2024-04-15T15:17:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2024-04-15T15:17:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2024-04-15T15:17:39.881 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2024-04-15T15:17:39.881 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2024-04-15T15:17:39.881 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2024-04-15T15:17:39.881 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2024-04-15T15:17:39.881 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2024-04-15T15:17:39.882 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2024-04-15T15:17:39.882 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2024-04-15T15:17:39.882 INFO:teuthology.orchestra.run.smithi049.stdout:[mds] 2024-04-15T15:17:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds = 20 2024-04-15T15:17:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds balancer = 20 2024-04-15T15:17:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-04-15T15:17:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug frag = True 2024-04-15T15:17:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug scatterstat = True 2024-04-15T15:17:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: mds op complaint time = 180 2024-04-15T15:17:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: mds verify scatter = True 2024-04-15T15:17:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2024-04-15T15:17:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2024-04-15T15:17:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2024-04-15T15:17:39.915 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-15T15:17:39.915 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-15T15:17:40.076 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-15T15:17:40.077 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:40.077 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-15T15:17:40.126 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:40.126 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-15T15:17:40.227 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:17:40.227 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:40.227 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-15T15:17:40.227 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-15T15:17:40.401 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:17:40.401 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:17:40.502 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:17:40.502 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:17:40.502 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-15T15:17:40.502 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-15T15:17:40.656 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:17:40.656 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:17:40.750 INFO:tasks.ceph:Setting up mon nodes... 2024-04-15T15:17:40.750 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-15T15:17:40.751 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-15T15:17:40.964 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-15T15:17:40.975 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-15T15:17:41.149 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-15T15:17:41.160 INFO:tasks.ceph:Setting up mds nodes... 2024-04-15T15:17:41.160 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-15T15:17:41.366 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-15T15:17:41.376 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-15T15:17:41.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-15T15:17:41.674 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-15T15:17:41.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-15T15:17:41.793 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-15T15:17:41.976 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-15T15:17:41.986 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-15T15:17:42.076 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-15T15:17:42.250 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-15T15:17:42.260 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-15T15:17:42.341 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-15T15:17:42.342 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-15T15:17:42.488 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-15T15:17:42.546 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-15T15:17:42.688 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-15T15:17:42.746 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-15T15:17:42.879 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-15T15:17:42.928 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-15T15:17:43.048 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-15T15:17:43.109 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-15T15:17:43.110 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-15T15:17:43.110 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-15T15:17:43.219 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:17:43.219 INFO:tasks.ceph:role: osd.0 2024-04-15T15:17:43.219 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:17:43.219 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-15T15:17:43.319 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:17:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:17:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:17:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:17:43.320 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:17:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:17:43.320 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:17:43.320 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:17:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:17:43.320 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:17:43.353 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-04-15T15:17:43.359 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-04-15T15:17:43.359 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-15T15:17:43.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-15T15:17:43.550 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-15T15:17:43.641 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:17:43.641 INFO:tasks.ceph:role: osd.1 2024-04-15T15:17:43.641 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:17:43.641 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-15T15:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-04-15T15:17:43.781 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-04-15T15:17:43.781 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-15T15:17:43.896 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-15T15:17:43.991 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-15T15:17:44.094 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:17:44.094 INFO:tasks.ceph:role: osd.2 2024-04-15T15:17:44.094 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:17:44.094 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-15T15:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-04-15T15:17:44.239 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-04-15T15:17:44.240 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-15T15:17:44.369 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-15T15:17:44.467 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-15T15:17:44.558 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:17:44.558 INFO:tasks.ceph:role: osd.3 2024-04-15T15:17:44.558 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:17:44.559 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-15T15:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-04-15T15:17:44.693 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-04-15T15:17:44.693 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-15T15:17:44.808 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-15T15:17:44.914 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:17:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:45.117+0000 7fb017613640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-15T15:17:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:45.119+0000 7fb017613640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-15T15:17:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:45.119+0000 7fb017613640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-15T15:17:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:45.119+0000 7fb017613640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-15T15:17:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:45.119+0000 7fb017613640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-15T15:17:45.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:45.122+0000 7fb017613640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-15T15:17:48.115 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-15T15:17:48.217 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:17:48.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:48.372+0000 7f8442a0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-15T15:17:48.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:48.372+0000 7f8442a0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-15T15:17:48.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:48.372+0000 7f8442a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-15T15:17:48.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:48.373+0000 7f8442a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-15T15:17:48.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:48.373+0000 7f8442a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-15T15:17:48.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:48.375+0000 7f8442a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-15T15:17:51.603 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-15T15:17:51.701 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:17:51.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:51.850+0000 7f870df9a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-15T15:17:51.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:51.851+0000 7f870df9a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-15T15:17:51.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:51.851+0000 7f870df9a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-15T15:17:51.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:51.851+0000 7f870df9a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-15T15:17:51.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:51.851+0000 7f870df9a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-15T15:17:51.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:51.853+0000 7f870df9a640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-15T15:17:54.807 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-15T15:17:54.911 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:17:55.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:55.066+0000 7f5fada15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-15T15:17:55.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:55.066+0000 7f5fada15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-15T15:17:55.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:55.066+0000 7f5fada15640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-15T15:17:55.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:55.066+0000 7f5fada15640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-15T15:17:55.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:55.066+0000 7f5fada15640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-15T15:17:55.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:17:55.069+0000 7f5fada15640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-15T15:17:58.028 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-15T15:17:58.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-15T15:17:58.240 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:17:58.240 INFO:tasks.ceph:role: osd.4 2024-04-15T15:17:58.240 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:17:58.240 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-15T15:17:58.338 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:17:58.339 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:17:58.339 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:17:58.339 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:17:58.339 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:17:58.339 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:17:58.339 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:17:58.339 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:17:58.339 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:17:58.339 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:17:58.750 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-04-15T15:17:58.757 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-04-15T15:17:58.758 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-15T15:17:58.925 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-15T15:17:59.021 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-15T15:17:59.103 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:17:59.104 INFO:tasks.ceph:role: osd.5 2024-04-15T15:17:59.104 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:17:59.104 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-15T15:17:59.191 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:17:59.191 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:17:59.191 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:17:59.191 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:17:59.191 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:17:59.191 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:17:59.191 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:17:59.192 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:17:59.192 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:17:59.192 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:17:59.546 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-04-15T15:17:59.551 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-04-15T15:17:59.552 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-15T15:17:59.676 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-15T15:17:59.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-15T15:17:59.835 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:17:59.835 INFO:tasks.ceph:role: osd.6 2024-04-15T15:17:59.835 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:17:59.836 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-15T15:17:59.925 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:17:59.925 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:17:59.925 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:17:59.925 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:17:59.925 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:17:59.925 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:17:59.925 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:17:59.925 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:17:59.925 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:17:59.925 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:18:00.306 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-04-15T15:18:00.313 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-04-15T15:18:00.314 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-15T15:18:00.474 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-15T15:18:00.569 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-15T15:18:00.640 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:18:00.640 INFO:tasks.ceph:role: osd.7 2024-04-15T15:18:00.640 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:18:00.640 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-15T15:18:00.723 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:18:00.724 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:18:00.724 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:18:00.724 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:18:00.724 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:18:00.724 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:18:00.724 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:18:00.724 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:18:00.724 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:18:00.724 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:18:01.128 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-04-15T15:18:01.135 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-04-15T15:18:01.135 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-15T15:18:01.292 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-15T15:18:01.387 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:01.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:01.543+0000 7fa7fb947640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-15T15:18:01.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:01.544+0000 7fa7fb947640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-15T15:18:01.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:01.544+0000 7fa7fb947640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-15T15:18:01.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:01.544+0000 7fa7fb947640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-15T15:18:01.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:01.544+0000 7fa7fb947640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-15T15:18:01.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:01.547+0000 7fa7fb947640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-15T15:18:04.513 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-15T15:18:04.606 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:04.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:04.738+0000 7fa60319a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-15T15:18:04.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:04.738+0000 7fa60319a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-15T15:18:04.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:04.739+0000 7fa60319a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-15T15:18:04.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:04.739+0000 7fa60319a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-15T15:18:04.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:04.739+0000 7fa60319a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-15T15:18:04.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:04.741+0000 7fa60319a640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-15T15:18:06.999 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:18:07.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:18:07.709 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-15T15:18:07.802 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:07.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:07.941+0000 7fba6a410640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-15T15:18:07.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:07.941+0000 7fba6a410640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-15T15:18:07.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:07.942+0000 7fba6a410640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-15T15:18:07.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:07.942+0000 7fba6a410640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-15T15:18:07.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:07.942+0000 7fba6a410640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-15T15:18:07.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:07.944+0000 7fba6a410640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-15T15:18:11.185 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-15T15:18:11.282 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:11.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:11.419+0000 7f12d65e1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-15T15:18:11.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:11.420+0000 7f12d65e1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-15T15:18:11.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:11.420+0000 7f12d65e1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-15T15:18:11.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:11.420+0000 7f12d65e1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-15T15:18:11.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:11.420+0000 7f12d65e1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-15T15:18:11.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:11.422+0000 7f12d65e1640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-15T15:18:14.393 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-15T15:18:14.484 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-15T15:18:14.484 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:14.485 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-15T15:18:14.599 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:14.599 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-15T15:18:14.699 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:14.699 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-15T15:18:14.810 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:14.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-15T15:18:14.914 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:14.914 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-15T15:18:15.015 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:15.015 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-15T15:18:15.121 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:15.122 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-15T15:18:15.234 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:15.234 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-15T15:18:15.337 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:15.337 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-15T15:18:15.429 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:15.429 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-15T15:18:15.517 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:15.517 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-15T15:18:15.659 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:15.659 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-15T15:18:15.747 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:15.747 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-15T15:18:15.833 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:15.833 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-15T15:18:15.964 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:15.964 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-15T15:18:16.025 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:16.025 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-15T15:18:16.084 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:16.084 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-15T15:18:16.140 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:16.140 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-15T15:18:16.193 INFO:tasks.ceph:Adding keys to all mons... 2024-04-15T15:18:16.194 DEBUG:teuthology.orchestra.run.smithi049:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-15T15:18:16.201 DEBUG:teuthology.orchestra.run.smithi107:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.y] 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAURR1mhIt2ORAAxgHgedKRzpFt/ARWPZjP3g== 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAVRR1m5WHRFRAAJXBz+oU3IyMQu/G5dsvabw== 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c] 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAVRR1mMUwqKBAAiSQOWs9ymyVAUExfk9Qclw== 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.0] 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAZRR1mt7wUBxAAXcBmE8sA/hGwdFvQ0HHMYA== 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.1] 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAcRR1mUT4+FhAAmdPWZRMVi+/75nhZVtfT9Q== 2024-04-15T15:18:16.286 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.2] 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAfRR1mYBvAMhAAyvQdSMQ0az5vuMX/lgzIwg== 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.3] 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAjRR1m7DcBBBAAfsRuWgRb5te0UjkDcd0IMQ== 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.x] 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAVRR1mKUPhCBAA9NPB6Z8xXRqJgEqs49pi7A== 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAVRR1m6EcuOhAAU3JqIQ65mBuJ8sG0DuIwog== 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d] 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAWRR1m1ynhDhAAHZ/86Gje4O++XTEh5HZV4g== 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.4] 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQApRR1mBz6AIBAAp73aMr+Lp1bAZyyqqBALGQ== 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.5] 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAsRR1m5tIQLBAAOAo1pTWsKMwiMwBF06pbjQ== 2024-04-15T15:18:16.287 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.6] 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAvRR1mIOcqOBAA4feE1ltGRQjt5h3NXU74Aw== 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.7] 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAzRR1mAPgPGRAAD2tksIXFnQdbN65/qYe5+A== 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAWRR1mkYQSHRAA0LcNsz2EApVHL/mhDGmMVg== 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout:[client.2] 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAWRR1mf7kDKRAACANrDIeJ2N6CqazlX4/DpQ== 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout:[client.1] 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAWRR1mn65aNBAAn12myNVRPC9WwpTHEEhtdg== 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout:[client.3] 2024-04-15T15:18:16.288 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAXRR1mX7HdAhAAOxlAH/My0eFkx9W5ZrDW0A== 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.y] 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAURR1mhIt2ORAAxgHgedKRzpFt/ARWPZjP3g== 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAVRR1m5WHRFRAAJXBz+oU3IyMQu/G5dsvabw== 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAVRR1mMUwqKBAAiSQOWs9ymyVAUExfk9Qclw== 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.0] 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAZRR1mt7wUBxAAXcBmE8sA/hGwdFvQ0HHMYA== 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.1] 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAcRR1mUT4+FhAAmdPWZRMVi+/75nhZVtfT9Q== 2024-04-15T15:18:16.302 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.2] 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAfRR1mYBvAMhAAyvQdSMQ0az5vuMX/lgzIwg== 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.3] 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAjRR1m7DcBBBAAfsRuWgRb5te0UjkDcd0IMQ== 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.x] 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAVRR1mKUPhCBAA9NPB6Z8xXRqJgEqs49pi7A== 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAVRR1m6EcuOhAAU3JqIQ65mBuJ8sG0DuIwog== 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAWRR1m1ynhDhAAHZ/86Gje4O++XTEh5HZV4g== 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.4] 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQApRR1mBz6AIBAAp73aMr+Lp1bAZyyqqBALGQ== 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.5] 2024-04-15T15:18:16.303 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAsRR1m5tIQLBAAOAo1pTWsKMwiMwBF06pbjQ== 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.6] 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAvRR1mIOcqOBAA4feE1ltGRQjt5h3NXU74Aw== 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.7] 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAzRR1mAPgPGRAAD2tksIXFnQdbN65/qYe5+A== 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAWRR1mkYQSHRAA0LcNsz2EApVHL/mhDGmMVg== 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout:[client.2] 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAWRR1mf7kDKRAACANrDIeJ2N6CqazlX4/DpQ== 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout:[client.1] 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAWRR1mn65aNBAAn12myNVRPC9WwpTHEEhtdg== 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout:[client.3] 2024-04-15T15:18:16.304 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAXRR1mX7HdAhAAOxlAH/My0eFkx9W5ZrDW0A== 2024-04-15T15:18:16.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:18:16.311 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:18:16.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:16.485 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:16.636 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:16.641 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:16.794 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:16.799 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:16.952 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:16.958 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:17.114 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:17.121 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:17.277 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:17.283 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:17.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:18:17.438 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:18:17.584 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:17.589 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:17.755 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:17.762 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:18:17.917 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:17.924 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:18.074 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:18.080 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:18.223 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:18.230 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:18.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:18.381 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:18:18.538 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:18:18.543 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:18:18.693 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:18:18.699 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:18:18.845 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:18:18.850 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:18:19.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:18:19.012 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:18:19.171 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-15T15:18:19.171 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-15T15:18:19.265 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-15T15:18:19.441 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-15T15:18:19.535 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-15T15:18:19.637 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-15T15:18:19.787 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-15T15:18:19.872 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-15T15:18:19.969 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-15T15:18:20.124 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-15T15:18:20.217 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:20.224 DEBUG:teuthology.orchestra.run.smithi107:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:18:20.277 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-15T15:18:20.277 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-15T15:18:20.278 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-15T15:18:20.283 INFO:tasks.ceph.mon.a:Started 2024-04-15T15:18:20.283 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-15T15:18:20.283 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-15T15:18:20.288 INFO:tasks.ceph.mon.b:Started 2024-04-15T15:18:20.288 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-15T15:18:20.288 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-15T15:18:20.293 INFO:tasks.ceph.mon.c:Started 2024-04-15T15:18:20.293 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-15T15:18:20.293 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-15T15:18:20.293 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-15T15:18:20.298 INFO:tasks.ceph.mgr.y:Started 2024-04-15T15:18:20.298 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-15T15:18:20.298 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-15T15:18:20.302 INFO:tasks.ceph.mgr.x:Started 2024-04-15T15:18:20.303 INFO:tasks.ceph:Setting crush tunables to default 2024-04-15T15:18:20.303 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd crush tunables default 2024-04-15T15:18:20.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.495+0000 7f020fe00640 1 Processor -- start 2024-04-15T15:18:20.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.495+0000 7f020fe00640 1 -- start start 2024-04-15T15:18:20.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.496+0000 7f020fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:20.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.496+0000 7f020fe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:20.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.496+0000 7f020fe00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02080ba4d0 con 0x7f02080d3270 2024-04-15T15:18:20.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.496+0000 7f020fe00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f020815f0a0 con 0x7f02080b3720 2024-04-15T15:18:20.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.496+0000 7f020fe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f02080ba280 con 0x7f020815ecc0 2024-04-15T15:18:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.496+0000 7f020d400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.496+0000 7f020d400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:43562/0 (socket says 172.21.15.49:43562) 2024-04-15T15:18:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.496+0000 7f020d400640 1 -- 172.21.15.49:0/605668833 learned_addr learned my addr 172.21.15.49:0/605668833 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.496+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.497+0000 7f020d400640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 msgr2=0x7f02080d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-15T15:18:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.497+0000 7f020d400640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 msgr2=0x7f02080d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.497+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.497+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-15T15:18:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.497+0000 7f020de00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 msgr2=0x7f02080cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-15T15:18:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.497+0000 7f020de00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 msgr2=0x7f02080cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:20.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.497+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:20.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.497+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-15T15:18:20.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.698+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:20.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.699+0000 7f020d400640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 msgr2=0x7f02080d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-15T15:18:20.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.699+0000 7f020d400640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 msgr2=0x7f02080d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:20.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.699+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:20.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.699+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:20.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.699+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-15T15:18:20.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.699+0000 7f020de00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 msgr2=0x7f02080cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-15T15:18:20.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.699+0000 7f020de00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 msgr2=0x7f02080cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.699+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:20.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:20.699+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-15T15:18:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.100+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.101+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.101+0000 7f020d400640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 msgr2=0x7f02080d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-15T15:18:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.101+0000 7f020d400640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 msgr2=0x7f02080d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.101+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.101+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-15T15:18:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.101+0000 7f020de00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 msgr2=0x7f02080cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-15T15:18:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.101+0000 7f020de00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 msgr2=0x7f02080cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.101+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:21.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.101+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-15T15:18:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.903+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.903+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.903+0000 7f020d400640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 msgr2=0x7f02080d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-15T15:18:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.903+0000 7f020d400640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 msgr2=0x7f02080d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.903+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.903+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-15T15:18:21.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.903+0000 7f020de00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 msgr2=0x7f02080cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-15T15:18:21.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.903+0000 7f020de00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 msgr2=0x7f02080cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:21.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.903+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:21.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:21.904+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-15T15:18:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.496+0000 7f0206a00640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 msgr2=0x7f02080d2d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-15T15:18:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.496+0000 7f0206a00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f020815ecc0 0x7f02080d2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.496+0000 7f0206a00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 msgr2=0x7f02080cf6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-15T15:18:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.496+0000 7f0206a00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.49:3300/0 conn(0x7f02080b3720 0x7f02080cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.496+0000 7f0206a00640 1 -- 172.21.15.49:0/605668833 >> v1:172.21.15.49:6789/0 conn(0x7f02080d3270 legacy=0x7f02080b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.496+0000 7f0206a00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.496+0000 7f0206a00640 1 -- 172.21.15.49:0/605668833 --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02080ba7b0 con 0x7f01e80036e0 2024-04-15T15:18:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.496+0000 7f0206a00640 1 -- 172.21.15.49:0/605668833 --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f020815f0a0 con 0x7f01e8007350 2024-04-15T15:18:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.497+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.498+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 msgr2=0x7f01e8009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-15T15:18:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.498+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 msgr2=0x7f01e8009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.498+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.498+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-15T15:18:23.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.699+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.700+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 msgr2=0x7f01e8009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-15T15:18:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.700+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 msgr2=0x7f01e8009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.700+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:23.700+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-15T15:18:24.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:24.101+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:24.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:24.102+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 msgr2=0x7f01e8009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-15T15:18:24.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:24.102+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 msgr2=0x7f01e8009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:24.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:24.102+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:24.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:24.102+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-15T15:18:24.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:24.903+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:24.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:24.904+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 msgr2=0x7f01e8009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-15T15:18:24.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:24.904+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 msgr2=0x7f01e8009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:18:24.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:24.904+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:18:24.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:24.904+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-15T15:18:25.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.541+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207774908 0 0) 0x7f02080ba7b0 con 0x7f01e80036e0 2024-04-15T15:18:25.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.541+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01f0003520 con 0x7f01e80036e0 2024-04-15T15:18:25.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.542+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1344640430 0 0) 0x7f01f0003520 con 0x7f01e80036e0 2024-04-15T15:18:25.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.542+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 msgr2=0x7f01e8009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-15T15:18:25.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.542+0000 7f0207e00640 1 --2- 172.21.15.49:0/605668833 >> v2:172.21.15.107:3300/0 conn(0x7f01e8007350 0x7f01e8009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.543+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e800af70 con 0x7f01e80036e0 2024-04-15T15:18:25.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.543+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f01fc00f040 con 0x7f01e80036e0 2024-04-15T15:18:25.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.543+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 >> v1:172.21.15.107:6789/0 conn(0x7f01e80036e0 legacy=0x7f01e8005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:25.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.543+0000 7f020de00640 1 -- 172.21.15.49:0/605668833 reap_dead start 2024-04-15T15:18:25.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.543+0000 7f0207e00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 0x7f01f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:25.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.543+0000 7f0207e00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 0x7f01f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:25.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.543+0000 7f0207e00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f00053c0 0x7f01f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:25.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.543+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f020815f0a0 con 0x7f01f0003a00 2024-04-15T15:18:25.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.543+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f01f000c280 con 0x7f01f00053c0 2024-04-15T15:18:25.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.543+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f01f000c480 con 0x7f01f0004800 2024-04-15T15:18:25.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.544+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 0x7f01f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:25.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.544+0000 7f020de00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f00053c0 0x7f01f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:25.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.545+0000 7f020d400640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 0x7f01f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:25.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.545+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f00053c0 msgr2=0x7f01f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:25.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.545+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f00053c0 0x7f01f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.545+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 msgr2=0x7f01f0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:25.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.546+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 0x7f01f0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.546+0000 7f020ca00640 1 -- 172.21.15.49:0/605668833 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f0003520 con 0x7f01f0004800 2024-04-15T15:18:25.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.548+0000 7f020ca00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 0x7f01f0004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f01f4007c30 tx=0x7f01f0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15926c2b29ad9d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:25.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.548+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4032070 con 0x7f01f0004800 2024-04-15T15:18:25.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.548+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f402ca40 con 0x7f01f0004800 2024-04-15T15:18:25.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.548+0000 7f0207e00640 1 -- 172.21.15.49:0/605668833 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f40051d0 con 0x7f01f0004800 2024-04-15T15:18:25.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.550+0000 7f020fe00640 1 -- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 msgr2=0x7f01f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:25.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.550+0000 7f020fe00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 0x7f01f0004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f01f4007c30 tx=0x7f01f0005830 comp rx=0 tx=0).stop 2024-04-15T15:18:25.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.550+0000 7f020fe00640 1 -- 172.21.15.49:0/605668833 shutdown_connections 2024-04-15T15:18:25.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.550+0000 7f020fe00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 0x7f01f0004c80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.550+0000 7f020fe00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f00053c0 0x7f01f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.550+0000 7f020fe00640 1 --2- 172.21.15.49:0/605668833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 0x7f01f0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.550+0000 7f020fe00640 1 -- 172.21.15.49:0/605668833 >> 172.21.15.49:0/605668833 conn(0x7f0208073480 msgr2=0x7f0208073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:25.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.550+0000 7f020fe00640 1 -- 172.21.15.49:0/605668833 shutdown_connections 2024-04-15T15:18:25.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.550+0000 7f020fe00640 1 -- 172.21.15.49:0/605668833 wait complete. 2024-04-15T15:18:25.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.554+0000 7f020fe00640 1 Processor -- start 2024-04-15T15:18:25.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.554+0000 7f020fe00640 1 -- start start 2024-04-15T15:18:25.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.555+0000 7f020fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 0x7f020815f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:25.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.555+0000 7f020fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 0x7f020814be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:25.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.555+0000 7f020fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f00053c0 0x7f02080cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:25.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.555+0000 7f020fe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f02080b3c70 con 0x7f01f0003a00 2024-04-15T15:18:25.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.555+0000 7f020fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f02080b90d0 con 0x7f01f00053c0 2024-04-15T15:18:25.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.555+0000 7f020fe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f02080b3df0 con 0x7f01f0004800 2024-04-15T15:18:25.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.556+0000 7f020d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 0x7f020815f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:25.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.556+0000 7f020d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 0x7f020815f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51614/0 (socket says 172.21.15.49:51614) 2024-04-15T15:18:25.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.556+0000 7f020d400640 1 -- 172.21.15.49:0/266245592 learned_addr learned my addr 172.21.15.49:0/266245592 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:25.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.556+0000 7f020ca00640 1 --2- 172.21.15.49:0/266245592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 0x7f020814be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:25.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.556+0000 7f020de00640 1 --2- 172.21.15.49:0/266245592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f00053c0 0x7f02080cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:25.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.556+0000 7f020d400640 1 -- 172.21.15.49:0/266245592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 msgr2=0x7f020814be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:25.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.556+0000 7f020d400640 1 --2- 172.21.15.49:0/266245592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 0x7f020814be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.556+0000 7f020d400640 1 -- 172.21.15.49:0/266245592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f00053c0 msgr2=0x7f02080cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:25.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.556+0000 7f020d400640 1 --2- 172.21.15.49:0/266245592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f00053c0 0x7f02080cfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.556+0000 7f020d400640 1 -- 172.21.15.49:0/266245592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020815f990 con 0x7f01f0003a00 2024-04-15T15:18:25.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.557+0000 7f020d400640 1 --2- 172.21.15.49:0/266245592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 0x7f020815f220 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f01fc015700 tx=0x7f01fc015bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:25.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.558+0000 7f0205600640 1 -- 172.21.15.49:0/266245592 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc013a60 con 0x7f01f0003a00 2024-04-15T15:18:25.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.558+0000 7f0205600640 1 -- 172.21.15.49:0/266245592 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01fc013c00 con 0x7f01f0003a00 2024-04-15T15:18:25.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.558+0000 7f0205600640 1 -- 172.21.15.49:0/266245592 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc020480 con 0x7f01f0003a00 2024-04-15T15:18:25.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.558+0000 7f020fe00640 1 -- 172.21.15.49:0/266245592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020815fc70 con 0x7f01f0003a00 2024-04-15T15:18:25.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.558+0000 7f020fe00640 1 -- 172.21.15.49:0/266245592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0208160230 con 0x7f01f0003a00 2024-04-15T15:18:25.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.559+0000 7f0205600640 1 -- 172.21.15.49:0/266245592 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f01fc020620 con 0x7f01f0003a00 2024-04-15T15:18:25.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.559+0000 7f0205600640 1 -- 172.21.15.49:0/266245592 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f01fc0163d0 con 0x7f01f0003a00 2024-04-15T15:18:25.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.560+0000 7f020fe00640 1 -- 172.21.15.49:0/266245592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0208068c80 con 0x7f01f0003a00 2024-04-15T15:18:25.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.564+0000 7f0205600640 1 -- 172.21.15.49:0/266245592 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f01fc020af0 con 0x7f01f0003a00 2024-04-15T15:18:25.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.641+0000 7f020fe00640 1 -- 172.21.15.49:0/266245592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f02081605a0 con 0x7f01f0003a00 2024-04-15T15:18:25.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.645+0000 7f0205600640 1 -- 172.21.15.49:0/266245592 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f01fc020cd0 con 0x7f01f0003a00 2024-04-15T15:18:25.647 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2024-04-15T15:18:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.647+0000 7f020fe00640 1 -- 172.21.15.49:0/266245592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 msgr2=0x7f020815f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.647+0000 7f020fe00640 1 --2- 172.21.15.49:0/266245592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 0x7f020815f220 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f01fc015700 tx=0x7f01fc015bc0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.647+0000 7f020fe00640 1 -- 172.21.15.49:0/266245592 shutdown_connections 2024-04-15T15:18:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.647+0000 7f020fe00640 1 --2- 172.21.15.49:0/266245592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f0004800 0x7f020814be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.647+0000 7f020fe00640 1 --2- 172.21.15.49:0/266245592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f00053c0 0x7f02080cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.647+0000 7f020fe00640 1 --2- 172.21.15.49:0/266245592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01f0003a00 0x7f020815f220 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.647+0000 7f020fe00640 1 -- 172.21.15.49:0/266245592 >> 172.21.15.49:0/266245592 conn(0x7f0208073480 msgr2=0x7f02080d3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.647+0000 7f020fe00640 1 -- 172.21.15.49:0/266245592 shutdown_connections 2024-04-15T15:18:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:25.647+0000 7f020fe00640 1 -- 172.21.15.49:0/266245592 wait complete. 2024-04-15T15:18:25.666 INFO:tasks.ceph:check_enable_crimson: False 2024-04-15T15:18:25.667 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-15T15:18:25.667 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:25.667 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-15T15:18:25.762 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:25.762 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-15T15:18:25.904 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:25.904 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-15T15:18:26.000 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:18:26.000 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-15T15:18:26.092 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:26.092 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-15T15:18:26.135 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:26.134+0000 7f3748c11040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-15T15:18:26.193 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:26.193 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-15T15:18:26.201 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:26.200+0000 7f3748c11040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-15T15:18:26.288 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:26.289 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-15T15:18:26.382 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:18:26.382 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-15T15:18:26.473 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 13788510-cda8-477d-a73e-0fb27d72cdaa 0 2024-04-15T15:18:26.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.692+0000 7fd000a00640 1 Processor -- start 2024-04-15T15:18:26.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.692+0000 7fd000a00640 1 -- start start 2024-04-15T15:18:26.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.694+0000 7fd000a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcffc0d17f0 0x7fcffc0d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:26.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.694+0000 7fd000a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcffc0b4cd0 0x7fcffc0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:26.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.694+0000 7fd000a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcffc0d2160 con 0x7fcffc1463c0 2024-04-15T15:18:26.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.694+0000 7fd000a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fcffc0b0880 con 0x7fcffc0d17f0 2024-04-15T15:18:26.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.694+0000 7fd000a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fcffc0b0b20 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.694+0000 7fcffa800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcffc0b4cd0 0x7fcffc0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:26.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.694+0000 7fcffa800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcffc0b4cd0 0x7fcffc0b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41754/0 (socket says 172.21.15.107:41754) 2024-04-15T15:18:26.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.694+0000 7fcffa800640 1 -- 172.21.15.107:0/1388931243 learned_addr learned my addr 172.21.15.107:0/1388931243 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:26.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.695+0000 7fcff9400640 1 --2- 172.21.15.107:0/1388931243 >> v2:172.21.15.49:3300/0 conn(0x7fcffc0d17f0 0x7fcffc0d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:26.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.695+0000 7fcffa800640 1 -- 172.21.15.107:0/1388931243 >> v2:172.21.15.49:3300/0 conn(0x7fcffc0d17f0 msgr2=0x7fcffc0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:26.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.695+0000 7fcffa800640 1 --2- 172.21.15.107:0/1388931243 >> v2:172.21.15.49:3300/0 conn(0x7fcffc0d17f0 0x7fcffc0d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:26.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.695+0000 7fcffa800640 1 -- 172.21.15.107:0/1388931243 >> v1:172.21.15.49:6789/0 conn(0x7fcffc1463c0 legacy=0x7fcffc0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:26.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.695+0000 7fcffa800640 1 -- 172.21.15.107:0/1388931243 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc0b0e00 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.696+0000 7fcffa800640 1 --2- 172.21.15.107:0/1388931243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 0x7fcffc0b5130 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0b6bb0 tx=0x7fcff402bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=613bbef4adfdfc28 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:26.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.696+0000 7fcff8a00640 1 -- 172.21.15.107:0/1388931243 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff402ca80 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.696+0000 7fcff8a00640 1 -- 172.21.15.107:0/1388931243 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff402cc20 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.696+0000 7fcff8a00640 1 -- 172.21.15.107:0/1388931243 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40343f0 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.697+0000 7fd000a00640 1 -- 172.21.15.107:0/1388931243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 msgr2=0x7fcffc0b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:26.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.697+0000 7fd000a00640 1 --2- 172.21.15.107:0/1388931243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 0x7fcffc0b5130 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0b6bb0 tx=0x7fcff402bee0 comp rx=0 tx=0).stop 2024-04-15T15:18:26.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.697+0000 7fd000a00640 1 -- 172.21.15.107:0/1388931243 shutdown_connections 2024-04-15T15:18:26.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.697+0000 7fd000a00640 1 --2- 172.21.15.107:0/1388931243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 0x7fcffc0b5130 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:26.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.697+0000 7fd000a00640 1 --2- 172.21.15.107:0/1388931243 >> v2:172.21.15.49:3300/0 conn(0x7fcffc0d17f0 0x7fcffc0d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:26.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.697+0000 7fd000a00640 1 -- 172.21.15.107:0/1388931243 >> 172.21.15.107:0/1388931243 conn(0x7fcffc09aa40 msgr2=0x7fcffc09ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:26.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.697+0000 7fd000a00640 1 -- 172.21.15.107:0/1388931243 shutdown_connections 2024-04-15T15:18:26.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.697+0000 7fd000a00640 1 -- 172.21.15.107:0/1388931243 wait complete. 2024-04-15T15:18:26.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.700+0000 7fd000a00640 1 Processor -- start 2024-04-15T15:18:26.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.700+0000 7fd000a00640 1 -- start start 2024-04-15T15:18:26.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fd000a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 0x7fcffc15bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:26.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fd000a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0d17f0 0x7fcffc070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:26.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fd000a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcffc1463c0 0x7fcffc0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:26.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fd000a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc0b0640 con 0x7fcffc1463c0 2024-04-15T15:18:26.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fd000a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc0b04c0 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fd000a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fcffc0b6310 con 0x7fcffc0d17f0 2024-04-15T15:18:26.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fcff9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 0x7fcffc15bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:26.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fcff9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 0x7fcffc15bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41760/0 (socket says 172.21.15.107:41760) 2024-04-15T15:18:26.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fcff9e00640 1 -- 172.21.15.107:0/1707869015 learned_addr learned my addr 172.21.15.107:0/1707869015 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:26.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fcff9400640 1 --2- 172.21.15.107:0/1707869015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0d17f0 0x7fcffc070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:26.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fcffa800640 1 --2- 172.21.15.107:0/1707869015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcffc1463c0 0x7fcffc0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:26.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fcff9e00640 1 -- 172.21.15.107:0/1707869015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0d17f0 msgr2=0x7fcffc070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:26.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fcff9e00640 1 --2- 172.21.15.107:0/1707869015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0d17f0 0x7fcffc070ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:26.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fcff9e00640 1 -- 172.21.15.107:0/1707869015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcffc1463c0 msgr2=0x7fcffc0cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:26.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fcff9e00640 1 --2- 172.21.15.107:0/1707869015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcffc1463c0 0x7fcffc0cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:26.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.702+0000 7fcff9e00640 1 -- 172.21.15.107:0/1707869015 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc0713a0 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.703+0000 7fcff9e00640 1 --2- 172.21.15.107:0/1707869015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 0x7fcffc15bd90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00c430 tx=0x7fcfec00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:26.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.703+0000 7fcff2000640 1 -- 172.21.15.107:0/1707869015 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec011790 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.703+0000 7fcff2000640 1 -- 172.21.15.107:0/1707869015 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec017070 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.703+0000 7fcff2000640 1 -- 172.21.15.107:0/1707869015 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec011bb0 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.703+0000 7fd000a00640 1 -- 172.21.15.107:0/1707869015 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc0cfc80 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.703+0000 7fd000a00640 1 -- 172.21.15.107:0/1707869015 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc0d0520 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.703+0000 7fcff2000640 1 -- 172.21.15.107:0/1707869015 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcfec01c460 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.704+0000 7fcff2000640 1 -- 172.21.15.107:0/1707869015 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fcfec0127c0 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.704+0000 7fd000a00640 1 -- 172.21.15.107:0/1707869015 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcffc0d0070 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.707+0000 7fcff2000640 1 -- 172.21.15.107:0/1707869015 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcfec011d50 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.787+0000 7fd000a00640 1 -- 172.21.15.107:0/1707869015 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "13788510-cda8-477d-a73e-0fb27d72cdaa", "id": 0} v 0) -- 0x7fcffc068c80 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.796+0000 7fcff2000640 1 -- 172.21.15.107:0/1707869015 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "13788510-cda8-477d-a73e-0fb27d72cdaa", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fcfec0129c0 con 0x7fcffc0b4cd0 2024-04-15T15:18:26.798 INFO:teuthology.orchestra.run.smithi107.stdout:0 2024-04-15T15:18:26.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.797+0000 7fd000a00640 1 -- 172.21.15.107:0/1707869015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 msgr2=0x7fcffc15bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:26.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.797+0000 7fd000a00640 1 --2- 172.21.15.107:0/1707869015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 0x7fcffc15bd90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00c430 tx=0x7fcfec00c8f0 comp rx=0 tx=0).stop 2024-04-15T15:18:26.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.798+0000 7fd000a00640 1 -- 172.21.15.107:0/1707869015 shutdown_connections 2024-04-15T15:18:26.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.798+0000 7fd000a00640 1 --2- 172.21.15.107:0/1707869015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0d17f0 0x7fcffc070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:26.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.798+0000 7fd000a00640 1 --2- 172.21.15.107:0/1707869015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcffc1463c0 0x7fcffc0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:26.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.798+0000 7fd000a00640 1 --2- 172.21.15.107:0/1707869015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0b4cd0 0x7fcffc15bd90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:26.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.798+0000 7fd000a00640 1 -- 172.21.15.107:0/1707869015 >> 172.21.15.107:0/1707869015 conn(0x7fcffc09aa40 msgr2=0x7fcffc0b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:26.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.798+0000 7fd000a00640 1 -- 172.21.15.107:0/1707869015 shutdown_connections 2024-04-15T15:18:26.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:26.798+0000 7fd000a00640 1 -- 172.21.15.107:0/1707869015 wait complete. 2024-04-15T15:18:26.817 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 7c41c3df-3579-4ac0-a310-bd1ec982129a 1 2024-04-15T15:18:26.843 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:26.841+0000 7f3748c11040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-15T15:18:26.911 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:26.909+0000 7f3748c11040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-15T15:18:26.965 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:26.963+0000 7f3748c11040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-15T15:18:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.007+0000 7feea9a00640 1 Processor -- start 2024-04-15T15:18:27.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.008+0000 7feea9a00640 1 -- start start 2024-04-15T15:18:27.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.009+0000 7feea9a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea40b3720 0x7feea40b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.009+0000 7feea9a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea40ba4d0 con 0x7feea40b3c70 2024-04-15T15:18:27.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.009+0000 7feea9a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea40ba7b0 con 0x7feea415ecc0 2024-04-15T15:18:27.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.009+0000 7feea9a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7feea415f0a0 con 0x7feea40b3720 2024-04-15T15:18:27.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.009+0000 7feea2a00640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7feea415ecc0 0x7feea40d2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:59870/0 (socket says 172.21.15.107:59870) 2024-04-15T15:18:27.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.009+0000 7feea2a00640 1 -- 172.21.15.107:0/1182090440 learned_addr learned my addr 172.21.15.107:0/1182090440 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-04-15T15:18:27.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.009+0000 7fee9be00640 1 --2- 172.21.15.107:0/1182090440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea40b3720 0x7feea40b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.010+0000 7feea2000640 1 -- 172.21.15.107:0/1182090440 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3696988897 0 0) 0x7feea40ba7b0 con 0x7feea415ecc0 2024-04-15T15:18:27.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.010+0000 7fee9be00640 1 -- 172.21.15.107:0/1182090440 >> v1:172.21.15.107:6789/0 conn(0x7feea415ecc0 legacy=0x7feea40d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.010+0000 7fee9be00640 1 -- 172.21.15.107:0/1182090440 >> v1:172.21.15.49:6789/0 conn(0x7feea40b3c70 legacy=0x7feea40cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:27.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.010+0000 7fee9be00640 1 -- 172.21.15.107:0/1182090440 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea40b8df0 con 0x7feea40b3720 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.011+0000 7fee9be00640 1 --2- 172.21.15.107:0/1182090440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea40b3720 0x7feea40b95f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0097d0 tx=0x7fee8c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dd0631240f18845 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.011+0000 7feea2000640 1 -- 172.21.15.107:0/1182090440 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c035020 con 0x7feea40b3720 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.011+0000 7feea2000640 1 -- 172.21.15.107:0/1182090440 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c031800 con 0x7feea40b3720 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.011+0000 7feea2000640 1 -- 172.21.15.107:0/1182090440 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c031bd0 con 0x7feea40b3720 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.012+0000 7feea9a00640 1 -- 172.21.15.107:0/1182090440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea40b3720 msgr2=0x7feea40b95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.012+0000 7feea9a00640 1 --2- 172.21.15.107:0/1182090440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea40b3720 0x7feea40b95f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0097d0 tx=0x7fee8c02b8a0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.012+0000 7feea9a00640 1 -- 172.21.15.107:0/1182090440 shutdown_connections 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.012+0000 7feea9a00640 1 --2- 172.21.15.107:0/1182090440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea40b3720 0x7feea40b95f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.012+0000 7feea9a00640 1 -- 172.21.15.107:0/1182090440 >> 172.21.15.107:0/1182090440 conn(0x7feea4073480 msgr2=0x7feea4073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.012+0000 7feea9a00640 1 -- 172.21.15.107:0/1182090440 shutdown_connections 2024-04-15T15:18:27.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.012+0000 7feea9a00640 1 -- 172.21.15.107:0/1182090440 wait complete. 2024-04-15T15:18:27.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.014+0000 7feea9a00640 1 Processor -- start 2024-04-15T15:18:27.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.014+0000 7feea9a00640 1 -- start start 2024-04-15T15:18:27.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7feea9a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feea40b3720 0x7feea40c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7feea9a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feea40b3c70 0x7feea414d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7feea9a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea415ecc0 0x7feea4161b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7feea9a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feea40b8aa0 con 0x7feea40b3720 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7feea9a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feea40ba280 con 0x7feea40b3c70 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7feea9a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7feea40d3440 con 0x7feea415ecc0 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7feea3400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea415ecc0 0x7feea4161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7fee9be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feea40b3c70 0x7feea414d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7feea3400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea415ecc0 0x7feea4161b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:56226/0 (socket says 172.21.15.107:56226) 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7fee9be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feea40b3c70 0x7feea414d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41774/0 (socket says 172.21.15.107:41774) 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.015+0000 7feea3400640 1 -- 172.21.15.107:0/3503392492 learned_addr learned my addr 172.21.15.107:0/3503392492 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.016+0000 7feea2a00640 1 --2- 172.21.15.107:0/3503392492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feea40b3720 0x7feea40c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.016+0000 7feea3400640 1 -- 172.21.15.107:0/3503392492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feea40b3c70 msgr2=0x7feea414d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.016+0000 7feea3400640 1 --2- 172.21.15.107:0/3503392492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feea40b3c70 0x7feea414d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.016+0000 7feea3400640 1 -- 172.21.15.107:0/3503392492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feea40b3720 msgr2=0x7feea40c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.016+0000 7feea3400640 1 --2- 172.21.15.107:0/3503392492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feea40b3720 0x7feea40c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.016+0000 7feea3400640 1 -- 172.21.15.107:0/3503392492 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea415b1a0 con 0x7feea415ecc0 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.016+0000 7feea3400640 1 --2- 172.21.15.107:0/3503392492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea415ecc0 0x7feea4161b00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c780 tx=0x7fee9c00cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.016+0000 7fee9aa00640 1 -- 172.21.15.107:0/3503392492 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c004b60 con 0x7feea415ecc0 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.017+0000 7fee9aa00640 1 -- 172.21.15.107:0/3503392492 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9c004d00 con 0x7feea415ecc0 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.017+0000 7fee9aa00640 1 -- 172.21.15.107:0/3503392492 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00aa90 con 0x7feea415ecc0 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.017+0000 7feea9a00640 1 -- 172.21.15.107:0/3503392492 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea4162040 con 0x7feea415ecc0 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.017+0000 7feea9a00640 1 -- 172.21.15.107:0/3503392492 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feea415b440 con 0x7feea415ecc0 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.017+0000 7fee9aa00640 1 -- 172.21.15.107:0/3503392492 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee9c006b50 con 0x7feea415ecc0 2024-04-15T15:18:27.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.017+0000 7fee9aa00640 1 -- 172.21.15.107:0/3503392492 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fee9c0071a0 con 0x7feea415ecc0 2024-04-15T15:18:27.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.018+0000 7feea9a00640 1 -- 172.21.15.107:0/3503392492 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea406de30 con 0x7feea415ecc0 2024-04-15T15:18:27.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.021+0000 7fee9aa00640 1 -- 172.21.15.107:0/3503392492 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7feea406de30 con 0x7feea415ecc0 2024-04-15T15:18:27.032 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:27.030+0000 7f3748c11040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-15T15:18:27.075 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:27.073+0000 7f1eb8238040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-15T15:18:27.084 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:27.083+0000 7f3748c11040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-15T15:18:27.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.101+0000 7feea9a00640 1 -- 172.21.15.107:0/3503392492 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7c41c3df-3579-4ac0-a310-bd1ec982129a", "id": 1} v 0) -- 0x7feea4068c80 con 0x7feea415ecc0 2024-04-15T15:18:27.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.107+0000 7fee9aa00640 1 -- 172.21.15.107:0/3503392492 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7c41c3df-3579-4ac0-a310-bd1ec982129a", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fee9c020070 con 0x7feea415ecc0 2024-04-15T15:18:27.109 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-15T15:18:27.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.109+0000 7feea9a00640 1 -- 172.21.15.107:0/3503392492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea415ecc0 msgr2=0x7feea4161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.109+0000 7feea9a00640 1 --2- 172.21.15.107:0/3503392492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea415ecc0 0x7feea4161b00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c780 tx=0x7fee9c00cc40 comp rx=0 tx=0).stop 2024-04-15T15:18:27.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.109+0000 7feea9a00640 1 -- 172.21.15.107:0/3503392492 shutdown_connections 2024-04-15T15:18:27.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.109+0000 7feea9a00640 1 --2- 172.21.15.107:0/3503392492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feea415ecc0 0x7feea4161b00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.109+0000 7feea9a00640 1 --2- 172.21.15.107:0/3503392492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feea40b3c70 0x7feea414d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.109+0000 7feea9a00640 1 --2- 172.21.15.107:0/3503392492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feea40b3720 0x7feea40c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.109+0000 7feea9a00640 1 -- 172.21.15.107:0/3503392492 >> 172.21.15.107:0/3503392492 conn(0x7feea4073480 msgr2=0x7feea40d4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:27.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.109+0000 7feea9a00640 1 -- 172.21.15.107:0/3503392492 shutdown_connections 2024-04-15T15:18:27.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.109+0000 7feea9a00640 1 -- 172.21.15.107:0/3503392492 wait complete. 2024-04-15T15:18:27.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new e6fd6d93-b856-4958-a445-b88822692f6f 2 2024-04-15T15:18:27.145 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:27.143+0000 7f1eb8238040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-15T15:18:27.223 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:27.222+0000 7f3748c11040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-15T15:18:27.309 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:27.307+0000 7f3748c11040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-15T15:18:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.316+0000 7ff95be00640 1 Processor -- start 2024-04-15T15:18:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.317+0000 7ff95be00640 1 -- start start 2024-04-15T15:18:27.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.318+0000 7ff95be00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff95c0ba460 0x7ff95c0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.318+0000 7ff95be00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff95c0d1480 0x7ff95c0d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.318+0000 7ff95be00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff95c163400 con 0x7ff95c15eb40 2024-04-15T15:18:27.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.318+0000 7ff95be00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff95c162c20 con 0x7ff95c0d1480 2024-04-15T15:18:27.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.318+0000 7ff95be00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7ff95c162e40 con 0x7ff95c0ba460 2024-04-15T15:18:27.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.318+0000 7ff953e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff95c0ba460 0x7ff95c0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.318+0000 7ff953e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff95c0ba460 0x7ff95c0d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41782/0 (socket says 172.21.15.107:41782) 2024-04-15T15:18:27.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.318+0000 7ff953e00640 1 -- 172.21.15.107:0/3376747712 learned_addr learned my addr 172.21.15.107:0/3376747712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.319+0000 7ff95b400640 1 --2- 172.21.15.107:0/3376747712 >> v2:172.21.15.49:3300/0 conn(0x7ff95c0d1480 0x7ff95c0d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.319+0000 7ff953e00640 1 -- 172.21.15.107:0/3376747712 >> v2:172.21.15.49:3300/0 conn(0x7ff95c0d1480 msgr2=0x7ff95c0d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.319+0000 7ff953e00640 1 --2- 172.21.15.107:0/3376747712 >> v2:172.21.15.49:3300/0 conn(0x7ff95c0d1480 0x7ff95c0d33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.319+0000 7ff953e00640 1 -- 172.21.15.107:0/3376747712 >> v1:172.21.15.49:6789/0 conn(0x7ff95c15eb40 legacy=0x7ff95c162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.319+0000 7ff953e00640 1 -- 172.21.15.107:0/3376747712 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c163120 con 0x7ff95c0ba460 2024-04-15T15:18:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.320+0000 7ff953e00640 1 --2- 172.21.15.107:0/3376747712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0ba460 0x7ff95c0d0f40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0ceca0 tx=0x7ff94402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46e3f693251558c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.320+0000 7ff95a000640 1 -- 172.21.15.107:0/3376747712 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944035020 con 0x7ff95c0ba460 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.320+0000 7ff95a000640 1 -- 172.21.15.107:0/3376747712 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff944031870 con 0x7ff95c0ba460 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.320+0000 7ff95a000640 1 -- 172.21.15.107:0/3376747712 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944031c40 con 0x7ff95c0ba460 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.321+0000 7ff95be00640 1 -- 172.21.15.107:0/3376747712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0ba460 msgr2=0x7ff95c0d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.321+0000 7ff95be00640 1 --2- 172.21.15.107:0/3376747712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0ba460 0x7ff95c0d0f40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0ceca0 tx=0x7ff94402b910 comp rx=0 tx=0).stop 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.321+0000 7ff95be00640 1 -- 172.21.15.107:0/3376747712 shutdown_connections 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.321+0000 7ff95be00640 1 --2- 172.21.15.107:0/3376747712 >> v2:172.21.15.49:3300/0 conn(0x7ff95c0d1480 0x7ff95c0d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.321+0000 7ff95be00640 1 --2- 172.21.15.107:0/3376747712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0ba460 0x7ff95c0d0f40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.321+0000 7ff95be00640 1 -- 172.21.15.107:0/3376747712 >> 172.21.15.107:0/3376747712 conn(0x7ff95c06d3d0 msgr2=0x7ff95c073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.321+0000 7ff95be00640 1 -- 172.21.15.107:0/3376747712 shutdown_connections 2024-04-15T15:18:27.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.321+0000 7ff95be00640 1 -- 172.21.15.107:0/3376747712 wait complete. 2024-04-15T15:18:27.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.323+0000 7ff95be00640 1 Processor -- start 2024-04-15T15:18:27.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.323+0000 7ff95be00640 1 -- start start 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff95be00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff95c0ba460 0x7ff95c14e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff95be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0d1480 0x7ff95c0cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff95be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff95c15eb40 0x7ff95c0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff95be00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c0b7860 con 0x7ff95c15eb40 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff95be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c0d2ea0 con 0x7ff95c0d1480 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff95be00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff95c0d3900 con 0x7ff95c0ba460 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff953e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0d1480 0x7ff95c0cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff953e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0d1480 0x7ff95c0cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41788/0 (socket says 172.21.15.107:41788) 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff95aa00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff95c0ba460 0x7ff95c14e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff953e00640 1 -- 172.21.15.107:0/3452861676 learned_addr learned my addr 172.21.15.107:0/3452861676 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff95b400640 1 --2- 172.21.15.107:0/3452861676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff95c15eb40 0x7ff95c0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff953e00640 1 -- 172.21.15.107:0/3452861676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff95c0ba460 msgr2=0x7ff95c14e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff953e00640 1 --2- 172.21.15.107:0/3452861676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff95c0ba460 0x7ff95c14e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff953e00640 1 -- 172.21.15.107:0/3452861676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff95c15eb40 msgr2=0x7ff95c0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff953e00640 1 --2- 172.21.15.107:0/3452861676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff95c15eb40 0x7ff95c0b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff953e00640 1 -- 172.21.15.107:0/3452861676 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c0b4c10 con 0x7ff95c0d1480 2024-04-15T15:18:27.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff95b400640 1 --2- 172.21.15.107:0/3452861676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff95c15eb40 0x7ff95c0b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:27.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.324+0000 7ff95aa00640 1 --2- 172.21.15.107:0/3452861676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff95c0ba460 0x7ff95c14e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:18:27.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.325+0000 7ff953e00640 1 --2- 172.21.15.107:0/3452861676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0d1480 0x7ff95c0cdeb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff944000c00 tx=0x7ff9440046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:27.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.325+0000 7ff953400640 1 -- 172.21.15.107:0/3452861676 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9440443f0 con 0x7ff95c0d1480 2024-04-15T15:18:27.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.325+0000 7ff953400640 1 -- 172.21.15.107:0/3452861676 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff944044590 con 0x7ff95c0d1480 2024-04-15T15:18:27.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.325+0000 7ff95be00640 1 -- 172.21.15.107:0/3452861676 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95c0b4ef0 con 0x7ff95c0d1480 2024-04-15T15:18:27.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.325+0000 7ff953400640 1 -- 172.21.15.107:0/3452861676 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944044980 con 0x7ff95c0d1480 2024-04-15T15:18:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.325+0000 7ff95be00640 1 -- 172.21.15.107:0/3452861676 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95c0b5790 con 0x7ff95c0d1480 2024-04-15T15:18:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.325+0000 7ff953400640 1 -- 172.21.15.107:0/3452861676 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff94404b060 con 0x7ff95c0d1480 2024-04-15T15:18:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.325+0000 7ff953400640 1 -- 172.21.15.107:0/3452861676 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7ff9440307e0 con 0x7ff95c0d1480 2024-04-15T15:18:27.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.326+0000 7ff95be00640 1 -- 172.21.15.107:0/3452861676 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff95c0b50c0 con 0x7ff95c0d1480 2024-04-15T15:18:27.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.328+0000 7ff953400640 1 -- 172.21.15.107:0/3452861676 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff94403f070 con 0x7ff95c0d1480 2024-04-15T15:18:27.366 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:27.364+0000 7f3748c11040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-15T15:18:27.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.407+0000 7ff95be00640 1 -- 172.21.15.107:0/3452861676 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6fd6d93-b856-4958-a445-b88822692f6f", "id": 2} v 0) -- 0x7ff95c0755c0 con 0x7ff95c0d1480 2024-04-15T15:18:27.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.413+0000 7ff953400640 1 -- 172.21.15.107:0/3452861676 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6fd6d93-b856-4958-a445-b88822692f6f", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7ff944031730 con 0x7ff95c0d1480 2024-04-15T15:18:27.415 INFO:teuthology.orchestra.run.smithi107.stdout:2 2024-04-15T15:18:27.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.414+0000 7ff95be00640 1 -- 172.21.15.107:0/3452861676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0d1480 msgr2=0x7ff95c0cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.414+0000 7ff95be00640 1 --2- 172.21.15.107:0/3452861676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0d1480 0x7ff95c0cdeb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff944000c00 tx=0x7ff9440046c0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.414+0000 7ff95be00640 1 -- 172.21.15.107:0/3452861676 shutdown_connections 2024-04-15T15:18:27.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.414+0000 7ff95be00640 1 --2- 172.21.15.107:0/3452861676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff95c15eb40 0x7ff95c0b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.414+0000 7ff95be00640 1 --2- 172.21.15.107:0/3452861676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff95c0d1480 0x7ff95c0cdeb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.414+0000 7ff95be00640 1 --2- 172.21.15.107:0/3452861676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff95c0ba460 0x7ff95c14e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.414+0000 7ff95be00640 1 -- 172.21.15.107:0/3452861676 >> 172.21.15.107:0/3452861676 conn(0x7ff95c06d3d0 msgr2=0x7ff95c0caba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:27.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.414+0000 7ff95be00640 1 -- 172.21.15.107:0/3452861676 shutdown_connections 2024-04-15T15:18:27.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.414+0000 7ff95be00640 1 -- 172.21.15.107:0/3452861676 wait complete. 2024-04-15T15:18:27.430 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:27.429+0000 7f3748c11040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-15T15:18:27.432 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new eb799a7e-9f21-4a9d-8a9d-f374b58f8074 3 2024-04-15T15:18:27.489 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:27.487+0000 7f3748c11040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-15T15:18:27.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.660+0000 7fac8a200640 1 Processor -- start 2024-04-15T15:18:27.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.660+0000 7fac8a200640 1 -- start start 2024-04-15T15:18:27.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.662+0000 7fac8a200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fac8415ecc0 0x7fac840d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.662+0000 7fac8a200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac840ba4d0 con 0x7fac840d3270 2024-04-15T15:18:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.662+0000 7fac8a200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac840ba7b0 con 0x7fac840b3720 2024-04-15T15:18:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.662+0000 7fac8a200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fac8415f0a0 con 0x7fac8415ecc0 2024-04-15T15:18:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.662+0000 7fac83e00640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fac840b3720 0x7fac840cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:59876/0 (socket says 172.21.15.107:59876) 2024-04-15T15:18:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.662+0000 7fac83e00640 1 -- 172.21.15.107:0/556508551 learned_addr learned my addr 172.21.15.107:0/556508551 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-04-15T15:18:27.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.662+0000 7fac83400640 1 --2- 172.21.15.107:0/556508551 >> v2:172.21.15.49:3300/0 conn(0x7fac8415ecc0 0x7fac840d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593225948 0 0) 0x7fac840ba7b0 con 0x7fac840b3720 2024-04-15T15:18:27.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac68003520 con 0x7fac840b3720 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 643665075 0 0) 0x7fac68003520 con 0x7fac840b3720 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 >> v2:172.21.15.49:3300/0 conn(0x7fac8415ecc0 msgr2=0x7fac840d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 --2- 172.21.15.107:0/556508551 >> v2:172.21.15.49:3300/0 conn(0x7fac8415ecc0 0x7fac840d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 >> v1:172.21.15.49:6789/0 conn(0x7fac840d3270 legacy=0x7fac840b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac840b8df0 con 0x7fac840b3720 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fac7c0019f0 con 0x7fac840b3720 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 >> v1:172.21.15.107:6789/0 conn(0x7fac840b3720 legacy=0x7fac840cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac680039b0 0x7fac68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac68004800 0x7fac68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680051c0 0x7fac6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac8415f0a0 con 0x7fac68004800 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac6800c060 con 0x7fac680039b0 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fac6800c260 con 0x7fac680051c0 2024-04-15T15:18:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.663+0000 7fac83e00640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680051c0 0x7fac6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.664+0000 7fac83400640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac680039b0 0x7fac68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.664+0000 7fac7be00640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac68004800 0x7fac68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.664+0000 7fac83400640 1 -- 172.21.15.107:0/556508551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680051c0 msgr2=0x7fac6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:27.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.664+0000 7fac83400640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680051c0 0x7fac6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.664+0000 7fac83400640 1 -- 172.21.15.107:0/556508551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac68004800 msgr2=0x7fac68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:27.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.664+0000 7fac83400640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac68004800 0x7fac68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.664+0000 7fac83400640 1 -- 172.21.15.107:0/556508551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac68003520 con 0x7fac680039b0 2024-04-15T15:18:27.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.664+0000 7fac83e00640 1 -- 172.21.15.107:0/556508551 reap_dead start 2024-04-15T15:18:27.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.666+0000 7fac83400640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac680039b0 0x7fac68003dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac74009ac0 tx=0x7fac68005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8cf5847661ed584 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:27.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.666+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74039070 con 0x7fac680039b0 2024-04-15T15:18:27.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.666+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7402c6d0 con 0x7fac680039b0 2024-04-15T15:18:27.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.666+0000 7fac82a00640 1 -- 172.21.15.107:0/556508551 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74009690 con 0x7fac680039b0 2024-04-15T15:18:27.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.666+0000 7fac8a200640 1 -- 172.21.15.107:0/556508551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac680039b0 msgr2=0x7fac68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.666+0000 7fac8a200640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac680039b0 0x7fac68003dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac74009ac0 tx=0x7fac68005830 comp rx=0 tx=0).stop 2024-04-15T15:18:27.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.667+0000 7fac8a200640 1 -- 172.21.15.107:0/556508551 shutdown_connections 2024-04-15T15:18:27.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.667+0000 7fac8a200640 1 --2- 172.21.15.107:0/556508551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac680039b0 0x7fac68003dd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.667+0000 7fac8a200640 1 -- 172.21.15.107:0/556508551 >> 172.21.15.107:0/556508551 conn(0x7fac84073480 msgr2=0x7fac84073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:27.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.667+0000 7fac8a200640 1 -- 172.21.15.107:0/556508551 shutdown_connections 2024-04-15T15:18:27.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.667+0000 7fac8a200640 1 -- 172.21.15.107:0/556508551 wait complete. 2024-04-15T15:18:27.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.669+0000 7fac8a200640 1 Processor -- start 2024-04-15T15:18:27.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.669+0000 7fac8a200640 1 -- start start 2024-04-15T15:18:27.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac8a200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680039b0 0x7fac841f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac8a200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac84151850 0x7fac841616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac8a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac840c8410 0x7fac840c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac8a200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac840b3c70 con 0x7fac84151850 2024-04-15T15:18:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac8a200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac840ba280 con 0x7fac840c8410 2024-04-15T15:18:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac8a200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fac840b3df0 con 0x7fac680039b0 2024-04-15T15:18:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac83400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680039b0 0x7fac841f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac83400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680039b0 0x7fac841f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:56264/0 (socket says 172.21.15.107:56264) 2024-04-15T15:18:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac83400640 1 -- 172.21.15.107:0/3063584838 learned_addr learned my addr 172.21.15.107:0/3063584838 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac83e00640 1 --2- 172.21.15.107:0/3063584838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac840c8410 0x7fac840c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac7be00640 1 --2- 172.21.15.107:0/3063584838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac84151850 0x7fac841616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac83400640 1 -- 172.21.15.107:0/3063584838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac840c8410 msgr2=0x7fac840c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac83400640 1 --2- 172.21.15.107:0/3063584838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac840c8410 0x7fac840c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac83400640 1 -- 172.21.15.107:0/3063584838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac84151850 msgr2=0x7fac841616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac83400640 1 --2- 172.21.15.107:0/3063584838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac84151850 0x7fac841616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.670+0000 7fac83400640 1 -- 172.21.15.107:0/3063584838 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac84151cd0 con 0x7fac680039b0 2024-04-15T15:18:27.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.671+0000 7fac83400640 1 --2- 172.21.15.107:0/3063584838 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680039b0 0x7fac841f3dd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fac74037040 tx=0x7fac74004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:27.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.671+0000 7fac7b400640 1 -- 172.21.15.107:0/3063584838 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74039050 con 0x7fac680039b0 2024-04-15T15:18:27.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.671+0000 7fac7b400640 1 -- 172.21.15.107:0/3063584838 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac74045070 con 0x7fac680039b0 2024-04-15T15:18:27.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.671+0000 7fac7b400640 1 -- 172.21.15.107:0/3063584838 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74003a70 con 0x7fac680039b0 2024-04-15T15:18:27.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.671+0000 7fac8a200640 1 -- 172.21.15.107:0/3063584838 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac84160650 con 0x7fac680039b0 2024-04-15T15:18:27.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.671+0000 7fac8a200640 1 -- 172.21.15.107:0/3063584838 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fac84160c10 con 0x7fac680039b0 2024-04-15T15:18:27.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.671+0000 7fac7b400640 1 -- 172.21.15.107:0/3063584838 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac74003c10 con 0x7fac680039b0 2024-04-15T15:18:27.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.671+0000 7fac7b400640 1 -- 172.21.15.107:0/3063584838 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fac74006250 con 0x7fac680039b0 2024-04-15T15:18:27.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.672+0000 7fac8a200640 1 -- 172.21.15.107:0/3063584838 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac84161c30 con 0x7fac680039b0 2024-04-15T15:18:27.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.677+0000 7fac7b400640 1 -- 172.21.15.107:0/3063584838 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fac7404f020 con 0x7fac680039b0 2024-04-15T15:18:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.756+0000 7fac8a200640 1 -- 172.21.15.107:0/3063584838 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "eb799a7e-9f21-4a9d-8a9d-f374b58f8074", "id": 3} v 0) -- 0x7fac84068c80 con 0x7fac680039b0 2024-04-15T15:18:27.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.762+0000 7fac7b400640 1 -- 172.21.15.107:0/3063584838 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eb799a7e-9f21-4a9d-8a9d-f374b58f8074", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fac740064a0 con 0x7fac680039b0 2024-04-15T15:18:27.764 INFO:teuthology.orchestra.run.smithi107.stdout:3 2024-04-15T15:18:27.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.763+0000 7fac8a200640 1 -- 172.21.15.107:0/3063584838 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680039b0 msgr2=0x7fac841f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.763+0000 7fac8a200640 1 --2- 172.21.15.107:0/3063584838 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680039b0 0x7fac841f3dd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fac74037040 tx=0x7fac74004730 comp rx=0 tx=0).stop 2024-04-15T15:18:27.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.763+0000 7fac8a200640 1 -- 172.21.15.107:0/3063584838 shutdown_connections 2024-04-15T15:18:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.763+0000 7fac8a200640 1 --2- 172.21.15.107:0/3063584838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac840c8410 0x7fac840c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.763+0000 7fac8a200640 1 --2- 172.21.15.107:0/3063584838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac84151850 0x7fac841616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.763+0000 7fac8a200640 1 --2- 172.21.15.107:0/3063584838 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac680039b0 0x7fac841f3dd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.763+0000 7fac8a200640 1 -- 172.21.15.107:0/3063584838 >> 172.21.15.107:0/3063584838 conn(0x7fac84073480 msgr2=0x7fac840b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.764+0000 7fac8a200640 1 -- 172.21.15.107:0/3063584838 shutdown_connections 2024-04-15T15:18:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.764+0000 7fac8a200640 1 -- 172.21.15.107:0/3063584838 wait complete. 2024-04-15T15:18:27.781 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new ae9fb934-8b18-47d9-9ab8-44204f9feb47 4 2024-04-15T15:18:27.787 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:27.785+0000 7f1eb8238040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-15T15:18:27.855 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:27.854+0000 7f1eb8238040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-15T15:18:27.908 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:27.907+0000 7f1eb8238040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-15T15:18:27.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.962+0000 7f470ca00640 1 Processor -- start 2024-04-15T15:18:27.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.962+0000 7f470ca00640 1 -- start start 2024-04-15T15:18:27.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.963+0000 7f470ca00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f4708162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.963+0000 7f470ca00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f47080d1650 0x7f47080d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.963+0000 7f470ca00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4708163400 con 0x7f47080ba460 2024-04-15T15:18:27.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.963+0000 7f470ca00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f4708162e40 con 0x7f47080d1650 2024-04-15T15:18:27.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.963+0000 7f470ca00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f4708163120 con 0x7f470815eb40 2024-04-15T15:18:27.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.964+0000 7f4705e00640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f47080ba460 0x7f47080d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:59886/0 (socket says 172.21.15.107:59886) 2024-04-15T15:18:27.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.964+0000 7f4705e00640 1 -- 172.21.15.107:0/1633812982 learned_addr learned my addr 172.21.15.107:0/1633812982 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-04-15T15:18:27.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.964+0000 7f4706800640 1 --2- 172.21.15.107:0/1633812982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f4708162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.964+0000 7f4707200640 1 --2- 172.21.15.107:0/1633812982 >> v2:172.21.15.49:3300/0 conn(0x7f47080d1650 0x7f47080d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.964+0000 7f4705400640 1 -- 172.21.15.107:0/1633812982 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3725094188 0 0) 0x7f4708163400 con 0x7f47080ba460 2024-04-15T15:18:27.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.964+0000 7f4706800640 1 -- 172.21.15.107:0/1633812982 >> v2:172.21.15.49:3300/0 conn(0x7f47080d1650 msgr2=0x7f47080d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:27.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.964+0000 7f4706800640 1 --2- 172.21.15.107:0/1633812982 >> v2:172.21.15.49:3300/0 conn(0x7f47080d1650 0x7f47080d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.964+0000 7f4706800640 1 -- 172.21.15.107:0/1633812982 >> v1:172.21.15.107:6789/0 conn(0x7f47080ba460 legacy=0x7f47080d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.964+0000 7f4706800640 1 -- 172.21.15.107:0/1633812982 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47080d2bc0 con 0x7f470815eb40 2024-04-15T15:18:27.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.965+0000 7f4706800640 1 --2- 172.21.15.107:0/1633812982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f4708162510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f46f80097a0 tx=0x7f46f802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46441a27a9131497 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:27.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f4705400640 1 -- 172.21.15.107:0/1633812982 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f80315a0 con 0x7f470815eb40 2024-04-15T15:18:27.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f4705400640 1 -- 172.21.15.107:0/1633812982 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f8037070 con 0x7f470815eb40 2024-04-15T15:18:27.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f4705400640 1 -- 172.21.15.107:0/1633812982 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8031960 con 0x7f470815eb40 2024-04-15T15:18:27.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f470ca00640 1 -- 172.21.15.107:0/1633812982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 msgr2=0x7f4708162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f470ca00640 1 --2- 172.21.15.107:0/1633812982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f4708162510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f46f80097a0 tx=0x7f46f802b840 comp rx=0 tx=0).stop 2024-04-15T15:18:27.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f470ca00640 1 -- 172.21.15.107:0/1633812982 shutdown_connections 2024-04-15T15:18:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f470ca00640 1 --2- 172.21.15.107:0/1633812982 >> v2:172.21.15.49:3300/0 conn(0x7f47080d1650 0x7f47080d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f470ca00640 1 --2- 172.21.15.107:0/1633812982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f4708162510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f470ca00640 1 -- 172.21.15.107:0/1633812982 >> 172.21.15.107:0/1633812982 conn(0x7f470806d3d0 msgr2=0x7f4708073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f470ca00640 1 -- 172.21.15.107:0/1633812982 shutdown_connections 2024-04-15T15:18:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.966+0000 7f470ca00640 1 -- 172.21.15.107:0/1633812982 wait complete. 2024-04-15T15:18:27.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.968+0000 7f470ca00640 1 Processor -- start 2024-04-15T15:18:27.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.968+0000 7f470ca00640 1 -- start start 2024-04-15T15:18:27.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f470ca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47080ba460 0x7f470814aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f470ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47080d1650 0x7f47080d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f470ca00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f47080d4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:27.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f470ca00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4708162a50 con 0x7f47080ba460 2024-04-15T15:18:27.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f470ca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f47080b7860 con 0x7f47080d1650 2024-04-15T15:18:27.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f470ca00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f4708162bd0 con 0x7f470815eb40 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f4707200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f47080d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f4705e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47080d1650 0x7f47080d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f4707200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f47080d4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:56284/0 (socket says 172.21.15.107:56284) 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f4705e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47080d1650 0x7f47080d3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41806/0 (socket says 172.21.15.107:41806) 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f4707200640 1 -- 172.21.15.107:0/2441487249 learned_addr learned my addr 172.21.15.107:0/2441487249 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.969+0000 7f4706800640 1 --2- 172.21.15.107:0/2441487249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47080ba460 0x7f470814aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f4705e00640 1 -- 172.21.15.107:0/2441487249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 msgr2=0x7f47080d4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f4705e00640 1 --2- 172.21.15.107:0/2441487249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f47080d4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f4705e00640 1 -- 172.21.15.107:0/2441487249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47080ba460 msgr2=0x7f470814aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f4705e00640 1 --2- 172.21.15.107:0/2441487249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47080ba460 0x7f470814aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f4705e00640 1 -- 172.21.15.107:0/2441487249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470814d790 con 0x7f47080d1650 2024-04-15T15:18:27.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f4707200640 1 --2- 172.21.15.107:0/2441487249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f47080d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:18:27.973 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:27.969+0000 7f3748c11040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-15T15:18:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f4705e00640 1 --2- 172.21.15.107:0/2441487249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47080d1650 0x7f47080d3ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c450 tx=0x7f46f000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f46fea00640 1 -- 172.21.15.107:0/2441487249 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0018070 con 0x7f47080d1650 2024-04-15T15:18:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f46fea00640 1 -- 172.21.15.107:0/2441487249 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f00074e0 con 0x7f47080d1650 2024-04-15T15:18:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f470ca00640 1 -- 172.21.15.107:0/2441487249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470814b400 con 0x7f47080d1650 2024-04-15T15:18:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.970+0000 7f46fea00640 1 -- 172.21.15.107:0/2441487249 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0004a20 con 0x7f47080d1650 2024-04-15T15:18:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.971+0000 7f470ca00640 1 -- 172.21.15.107:0/2441487249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470814dd40 con 0x7f47080d1650 2024-04-15T15:18:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.971+0000 7f46fea00640 1 -- 172.21.15.107:0/2441487249 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f46f001b070 con 0x7f47080d1650 2024-04-15T15:18:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.971+0000 7f46fea00640 1 -- 172.21.15.107:0/2441487249 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f46f0004bc0 con 0x7f47080d1650 2024-04-15T15:18:27.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.972+0000 7f470ca00640 1 -- 172.21.15.107:0/2441487249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47080755c0 con 0x7f47080d1650 2024-04-15T15:18:27.975 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:27.974+0000 7f1eb8238040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-15T15:18:27.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:27.974+0000 7f46fea00640 1 -- 172.21.15.107:0/2441487249 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f46f0004e10 con 0x7f47080d1650 2024-04-15T15:18:28.028 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:28.026+0000 7f1eb8238040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-15T15:18:28.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.053+0000 7f470ca00640 1 -- 172.21.15.107:0/2441487249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ae9fb934-8b18-47d9-9ab8-44204f9feb47", "id": 4} v 0) -- 0x7f470815c810 con 0x7f47080d1650 2024-04-15T15:18:28.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.060+0000 7f46fea00640 1 -- 172.21.15.107:0/2441487249 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae9fb934-8b18-47d9-9ab8-44204f9feb47", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f46f001dbc0 con 0x7f47080d1650 2024-04-15T15:18:28.061 INFO:teuthology.orchestra.run.smithi107.stdout:4 2024-04-15T15:18:28.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.061+0000 7f470ca00640 1 -- 172.21.15.107:0/2441487249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47080d1650 msgr2=0x7f47080d3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.061+0000 7f470ca00640 1 --2- 172.21.15.107:0/2441487249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47080d1650 0x7f47080d3ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c450 tx=0x7f46f000c910 comp rx=0 tx=0).stop 2024-04-15T15:18:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.061+0000 7f470ca00640 1 -- 172.21.15.107:0/2441487249 shutdown_connections 2024-04-15T15:18:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.061+0000 7f470ca00640 1 --2- 172.21.15.107:0/2441487249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f470815eb40 0x7f47080d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.061+0000 7f470ca00640 1 --2- 172.21.15.107:0/2441487249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47080d1650 0x7f47080d3ed0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.061+0000 7f470ca00640 1 --2- 172.21.15.107:0/2441487249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47080ba460 0x7f470814aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.061+0000 7f470ca00640 1 -- 172.21.15.107:0/2441487249 >> 172.21.15.107:0/2441487249 conn(0x7f470806d3d0 msgr2=0x7f470815cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.061+0000 7f470ca00640 1 -- 172.21.15.107:0/2441487249 shutdown_connections 2024-04-15T15:18:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.062+0000 7f470ca00640 1 -- 172.21.15.107:0/2441487249 wait complete. 2024-04-15T15:18:28.080 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new f500a3b5-7007-47b2-8d18-3ceecadaa55c 5 2024-04-15T15:18:28.143 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:28.141+0000 7f3748c11040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-15T15:18:28.168 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:28.166+0000 7f1eb8238040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-15T15:18:28.253 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:28.250+0000 7f1eb8238040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-15T15:18:28.290 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:28.288+0000 7f3748c11040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-15T15:18:28.309 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:28.307+0000 7f1eb8238040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-15T15:18:28.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.312+0000 7f80b7e00640 1 Processor -- start 2024-04-15T15:18:28.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.312+0000 7f80b7e00640 1 -- start start 2024-04-15T15:18:28.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.313+0000 7f80b7e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b806b930 0x7f80b815f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.314+0000 7f80b7e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b815fe70 con 0x7f80b81609a0 2024-04-15T15:18:28.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.314+0000 7f80b7e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b806c820 con 0x7f80b80b1b00 2024-04-15T15:18:28.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.314+0000 7f80b7e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f80b815c950 con 0x7f80b806b930 2024-04-15T15:18:28.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.314+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b806b930 0x7f80b815f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.314+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b806b930 0x7f80b815f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:56294/0 (socket says 172.21.15.107:56294) 2024-04-15T15:18:28.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.314+0000 7f80b6a00640 1 -- 172.21.15.107:0/2956112889 learned_addr learned my addr 172.21.15.107:0/2956112889 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:28.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.314+0000 7f80b6a00640 1 -- 172.21.15.107:0/2956112889 >> v1:172.21.15.107:6789/0 conn(0x7f80b80b1b00 legacy=0x7f80b806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.314+0000 7f80b6a00640 1 -- 172.21.15.107:0/2956112889 >> v1:172.21.15.49:6789/0 conn(0x7f80b81609a0 legacy=0x7f80b8160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.314+0000 7f80b6a00640 1 -- 172.21.15.107:0/2956112889 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b80b1f90 con 0x7f80b806b930 2024-04-15T15:18:28.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.314+0000 7f80b5600640 1 -- 172.21.15.107:0/2956112889 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316198787 0 0) 0x7f80b806c820 con 0x7f80b80b1b00 2024-04-15T15:18:28.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.316+0000 7f80b6a00640 1 --2- 172.21.15.107:0/2956112889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b806b930 0x7f80b815f930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f80a80098a0 tx=0x7f80a802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95094daed776383 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:28.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.316+0000 7f80b5600640 1 -- 172.21.15.107:0/2956112889 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8034070 con 0x7f80b806b930 2024-04-15T15:18:28.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.316+0000 7f80b5600640 1 -- 172.21.15.107:0/2956112889 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a8030840 con 0x7f80b806b930 2024-04-15T15:18:28.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.316+0000 7f80b5600640 1 -- 172.21.15.107:0/2956112889 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8030c10 con 0x7f80b806b930 2024-04-15T15:18:28.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.316+0000 7f80b7e00640 1 -- 172.21.15.107:0/2956112889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b806b930 msgr2=0x7f80b815f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.316+0000 7f80b7e00640 1 --2- 172.21.15.107:0/2956112889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b806b930 0x7f80b815f930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f80a80098a0 tx=0x7f80a802b980 comp rx=0 tx=0).stop 2024-04-15T15:18:28.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.317+0000 7f80b7e00640 1 -- 172.21.15.107:0/2956112889 shutdown_connections 2024-04-15T15:18:28.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.317+0000 7f80b7e00640 1 --2- 172.21.15.107:0/2956112889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b806b930 0x7f80b815f930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.317+0000 7f80b7e00640 1 -- 172.21.15.107:0/2956112889 >> 172.21.15.107:0/2956112889 conn(0x7f80b80b69c0 msgr2=0x7f80b80b6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:28.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.317+0000 7f80b7e00640 1 -- 172.21.15.107:0/2956112889 shutdown_connections 2024-04-15T15:18:28.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.317+0000 7f80b7e00640 1 -- 172.21.15.107:0/2956112889 wait complete. 2024-04-15T15:18:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.319+0000 7f80b7e00640 1 Processor -- start 2024-04-15T15:18:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.319+0000 7f80b7e00640 1 -- start start 2024-04-15T15:18:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.320+0000 7f80b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b806b930 0x7f80b8157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.320+0000 7f80b7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b81609a0 0x7f80b81e65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.320+0000 7f80b7e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b81e8d60 0x7f80b80cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.320+0000 7f80b7e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81601a0 con 0x7f80b81609a0 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.320+0000 7f80b7e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8160480 con 0x7f80b806b930 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.320+0000 7f80b7e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f80b80b2550 con 0x7f80b81e8d60 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.320+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b806b930 0x7f80b8157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.321+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b806b930 0x7f80b8157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41814/0 (socket says 172.21.15.107:41814) 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.321+0000 7f80b7400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b81e8d60 0x7f80b80cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.321+0000 7f80b6a00640 1 -- 172.21.15.107:0/1440788720 learned_addr learned my addr 172.21.15.107:0/1440788720 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.321+0000 7f80b6000640 1 --2- 172.21.15.107:0/1440788720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b81609a0 0x7f80b81e65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.321+0000 7f80b6a00640 1 -- 172.21.15.107:0/1440788720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b81e8d60 msgr2=0x7f80b80cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.321+0000 7f80b6a00640 1 --2- 172.21.15.107:0/1440788720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b81e8d60 0x7f80b80cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.321+0000 7f80b6a00640 1 -- 172.21.15.107:0/1440788720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b81609a0 msgr2=0x7f80b81e65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.321+0000 7f80b6a00640 1 --2- 172.21.15.107:0/1440788720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b81609a0 0x7f80b81e65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.321+0000 7f80b6a00640 1 -- 172.21.15.107:0/1440788720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b8157a50 con 0x7f80b806b930 2024-04-15T15:18:28.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.321+0000 7f80b6a00640 1 --2- 172.21.15.107:0/1440788720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b806b930 0x7f80b8157510 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80a80098a0 tx=0x7f80a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:28.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.322+0000 7f80aea00640 1 -- 172.21.15.107:0/1440788720 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8034040 con 0x7f80b806b930 2024-04-15T15:18:28.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.322+0000 7f80aea00640 1 -- 172.21.15.107:0/1440788720 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a803ab10 con 0x7f80b806b930 2024-04-15T15:18:28.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.322+0000 7f80aea00640 1 -- 172.21.15.107:0/1440788720 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8004030 con 0x7f80b806b930 2024-04-15T15:18:28.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.322+0000 7f80b7e00640 1 -- 172.21.15.107:0/1440788720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b81e91f0 con 0x7f80b806b930 2024-04-15T15:18:28.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.322+0000 7f80b7e00640 1 -- 172.21.15.107:0/1440788720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b81e97b0 con 0x7f80b806b930 2024-04-15T15:18:28.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.322+0000 7f80aea00640 1 -- 172.21.15.107:0/1440788720 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80a804b060 con 0x7f80b806b930 2024-04-15T15:18:28.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.322+0000 7f80aea00640 1 -- 172.21.15.107:0/1440788720 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f80a80041d0 con 0x7f80b806b930 2024-04-15T15:18:28.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.323+0000 7f80b7e00640 1 -- 172.21.15.107:0/1440788720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b80755c0 con 0x7f80b806b930 2024-04-15T15:18:28.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.325+0000 7f80aea00640 1 -- 172.21.15.107:0/1440788720 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f80a803f070 con 0x7f80b806b930 2024-04-15T15:18:28.347 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:28.345+0000 7f3748c11040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-15T15:18:28.381 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:28.379+0000 7f1eb8238040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-15T15:18:28.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.413+0000 7f80b7e00640 1 -- 172.21.15.107:0/1440788720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f500a3b5-7007-47b2-8d18-3ceecadaa55c", "id": 5} v 0) -- 0x7f80b80d1280 con 0x7f80b806b930 2024-04-15T15:18:28.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.421+0000 7f80aea00640 1 -- 172.21.15.107:0/1440788720 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f500a3b5-7007-47b2-8d18-3ceecadaa55c", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f80a80044a0 con 0x7f80b806b930 2024-04-15T15:18:28.423 INFO:teuthology.orchestra.run.smithi107.stdout:5 2024-04-15T15:18:28.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.422+0000 7f80b7e00640 1 -- 172.21.15.107:0/1440788720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b806b930 msgr2=0x7f80b8157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.422+0000 7f80b7e00640 1 --2- 172.21.15.107:0/1440788720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b806b930 0x7f80b8157510 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80a80098a0 tx=0x7f80a80046c0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.422+0000 7f80b7e00640 1 -- 172.21.15.107:0/1440788720 shutdown_connections 2024-04-15T15:18:28.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.422+0000 7f80b7e00640 1 --2- 172.21.15.107:0/1440788720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80b81e8d60 0x7f80b80cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.422+0000 7f80b7e00640 1 --2- 172.21.15.107:0/1440788720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80b81609a0 0x7f80b81e65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.422+0000 7f80b7e00640 1 --2- 172.21.15.107:0/1440788720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b806b930 0x7f80b8157510 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.422+0000 7f80b7e00640 1 -- 172.21.15.107:0/1440788720 >> 172.21.15.107:0/1440788720 conn(0x7f80b80b69c0 msgr2=0x7f80b80ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.423+0000 7f80b7e00640 1 -- 172.21.15.107:0/1440788720 shutdown_connections 2024-04-15T15:18:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.423+0000 7f80b7e00640 1 -- 172.21.15.107:0/1440788720 wait complete. 2024-04-15T15:18:28.437 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:28.436+0000 7f1eb8238040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-15T15:18:28.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 84196c37-bce8-4069-99b8-7368d0798b9f 6 2024-04-15T15:18:28.483 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:28.481+0000 7f3748c11040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:18:28.601 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:28.599+0000 7f3748c11040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-15T15:18:28.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.672+0000 7fc5ee200640 1 Processor -- start 2024-04-15T15:18:28.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.672+0000 7fc5ee200640 1 -- start start 2024-04-15T15:18:28.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.674+0000 7fc5ee200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc5e815eb40 0x7fc5e8162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.674+0000 7fc5ee200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc5e80ba460 0x7fc5e80d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.674+0000 7fc5ee200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5e8163400 con 0x7fc5e80d1480 2024-04-15T15:18:28.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.674+0000 7fc5ee200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc5e8162e40 con 0x7fc5e815eb40 2024-04-15T15:18:28.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.674+0000 7fc5ee200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fc5e8163120 con 0x7fc5e80ba460 2024-04-15T15:18:28.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.674+0000 7fc5e7e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc5e80ba460 0x7fc5e80d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.674+0000 7fc5e7e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc5e80ba460 0x7fc5e80d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41818/0 (socket says 172.21.15.107:41818) 2024-04-15T15:18:28.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.674+0000 7fc5e7e00640 1 -- 172.21.15.107:0/292842551 learned_addr learned my addr 172.21.15.107:0/292842551 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:28.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.674+0000 7fc5ed800640 1 --1- 172.21.15.107:0/292842551 >> v1:172.21.15.107:6789/0 conn(0x7fc5e80d1480 0x7fc5e80b61b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:59906/0 (socket says 172.21.15.107:59906) 2024-04-15T15:18:28.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.674+0000 7fc5ece00640 1 --2- 172.21.15.107:0/292842551 >> v2:172.21.15.49:3300/0 conn(0x7fc5e815eb40 0x7fc5e8162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.675+0000 7fc5e7e00640 1 -- 172.21.15.107:0/292842551 >> v2:172.21.15.49:3300/0 conn(0x7fc5e815eb40 msgr2=0x7fc5e8162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:28.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.675+0000 7fc5e7e00640 1 --2- 172.21.15.107:0/292842551 >> v2:172.21.15.49:3300/0 conn(0x7fc5e815eb40 0x7fc5e8162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.675+0000 7fc5e7e00640 1 -- 172.21.15.107:0/292842551 >> v1:172.21.15.107:6789/0 conn(0x7fc5e80d1480 legacy=0x7fc5e80b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.675+0000 7fc5e7e00640 1 -- 172.21.15.107:0/292842551 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e80d2bc0 con 0x7fc5e80ba460 2024-04-15T15:18:28.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.676+0000 7fc5e7e00640 1 --2- 172.21.15.107:0/292842551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 0x7fc5e80d0f40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80ceca0 tx=0x7fc5d402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db0145f8f551efc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:28.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.676+0000 7fc5e7400640 1 -- 172.21.15.107:0/292842551 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4035020 con 0x7fc5e80ba460 2024-04-15T15:18:28.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.676+0000 7fc5e7400640 1 -- 172.21.15.107:0/292842551 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d4031870 con 0x7fc5e80ba460 2024-04-15T15:18:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.676+0000 7fc5e7400640 1 -- 172.21.15.107:0/292842551 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4031c40 con 0x7fc5e80ba460 2024-04-15T15:18:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.677+0000 7fc5ee200640 1 -- 172.21.15.107:0/292842551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 msgr2=0x7fc5e80d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.677+0000 7fc5ee200640 1 --2- 172.21.15.107:0/292842551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 0x7fc5e80d0f40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80ceca0 tx=0x7fc5d402b910 comp rx=0 tx=0).stop 2024-04-15T15:18:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.677+0000 7fc5ee200640 1 -- 172.21.15.107:0/292842551 shutdown_connections 2024-04-15T15:18:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.677+0000 7fc5ee200640 1 --2- 172.21.15.107:0/292842551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 0x7fc5e80d0f40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.677+0000 7fc5ee200640 1 --2- 172.21.15.107:0/292842551 >> v2:172.21.15.49:3300/0 conn(0x7fc5e815eb40 0x7fc5e8162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.677+0000 7fc5ee200640 1 -- 172.21.15.107:0/292842551 >> 172.21.15.107:0/292842551 conn(0x7fc5e806d3d0 msgr2=0x7fc5e8073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.677+0000 7fc5ee200640 1 -- 172.21.15.107:0/292842551 shutdown_connections 2024-04-15T15:18:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.677+0000 7fc5ee200640 1 -- 172.21.15.107:0/292842551 wait complete. 2024-04-15T15:18:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.679+0000 7fc5ee200640 1 Processor -- start 2024-04-15T15:18:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.679+0000 7fc5ee200640 1 -- start start 2024-04-15T15:18:28.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5ee200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 0x7fc5e814b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5ee200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5e80d1480 0x7fc5e81f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5ee200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e815eb40 0x7fc5e80b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5ee200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e80d43d0 con 0x7fc5e80d1480 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5ee200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e80b7860 con 0x7fc5e80ba460 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5ee200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e80d4550 con 0x7fc5e815eb40 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5ece00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 0x7fc5e814b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5ece00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 0x7fc5e814b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41828/0 (socket says 172.21.15.107:41828) 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5ece00640 1 -- 172.21.15.107:0/3317254996 learned_addr learned my addr 172.21.15.107:0/3317254996 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5ed800640 1 --2- 172.21.15.107:0/3317254996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e815eb40 0x7fc5e80b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.680+0000 7fc5e7e00640 1 --2- 172.21.15.107:0/3317254996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5e80d1480 0x7fc5e81f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5ece00640 1 -- 172.21.15.107:0/3317254996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e815eb40 msgr2=0x7fc5e80b8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5ece00640 1 --2- 172.21.15.107:0/3317254996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e815eb40 0x7fc5e80b8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5ece00640 1 -- 172.21.15.107:0/3317254996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5e80d1480 msgr2=0x7fc5e81f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5ece00640 1 --2- 172.21.15.107:0/3317254996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5e80d1480 0x7fc5e81f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5ece00640 1 -- 172.21.15.107:0/3317254996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e81f5090 con 0x7fc5e80ba460 2024-04-15T15:18:28.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5ece00640 1 --2- 172.21.15.107:0/3317254996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 0x7fc5e814b560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00e870 tx=0x7fc5dc00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:28.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5e4c00640 1 -- 172.21.15.107:0/3317254996 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc017070 con 0x7fc5e80ba460 2024-04-15T15:18:28.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5e4c00640 1 -- 172.21.15.107:0/3317254996 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5dc00c940 con 0x7fc5e80ba460 2024-04-15T15:18:28.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5e4c00640 1 -- 172.21.15.107:0/3317254996 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00cd30 con 0x7fc5e80ba460 2024-04-15T15:18:28.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5ee200640 1 -- 172.21.15.107:0/3317254996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e81612d0 con 0x7fc5e80ba460 2024-04-15T15:18:28.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.681+0000 7fc5ee200640 1 -- 172.21.15.107:0/3317254996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e8161890 con 0x7fc5e80ba460 2024-04-15T15:18:28.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.682+0000 7fc5e4c00640 1 -- 172.21.15.107:0/3317254996 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc5dc01a070 con 0x7fc5e80ba460 2024-04-15T15:18:28.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.682+0000 7fc5e4c00640 1 -- 172.21.15.107:0/3317254996 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fc5dc0064d0 con 0x7fc5e80ba460 2024-04-15T15:18:28.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.682+0000 7fc5ee200640 1 -- 172.21.15.107:0/3317254996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e80755c0 con 0x7fc5e80ba460 2024-04-15T15:18:28.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.685+0000 7fc5e4c00640 1 -- 172.21.15.107:0/3317254996 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc5dc006770 con 0x7fc5e80ba460 2024-04-15T15:18:28.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.764+0000 7fc5ee200640 1 -- 172.21.15.107:0/3317254996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "84196c37-bce8-4069-99b8-7368d0798b9f", "id": 6} v 0) -- 0x7fc5e80b4210 con 0x7fc5e80ba460 2024-04-15T15:18:28.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.775+0000 7fc5e4c00640 1 -- 172.21.15.107:0/3317254996 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "84196c37-bce8-4069-99b8-7368d0798b9f", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fc5dc005990 con 0x7fc5e80ba460 2024-04-15T15:18:28.776 INFO:teuthology.orchestra.run.smithi107.stdout:6 2024-04-15T15:18:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.776+0000 7fc5ee200640 1 -- 172.21.15.107:0/3317254996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 msgr2=0x7fc5e814b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.776+0000 7fc5ee200640 1 --2- 172.21.15.107:0/3317254996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 0x7fc5e814b560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00e870 tx=0x7fc5dc00ed30 comp rx=0 tx=0).stop 2024-04-15T15:18:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.776+0000 7fc5ee200640 1 -- 172.21.15.107:0/3317254996 shutdown_connections 2024-04-15T15:18:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.776+0000 7fc5ee200640 1 --2- 172.21.15.107:0/3317254996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e815eb40 0x7fc5e80b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.776+0000 7fc5ee200640 1 --2- 172.21.15.107:0/3317254996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5e80d1480 0x7fc5e81f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.776+0000 7fc5ee200640 1 --2- 172.21.15.107:0/3317254996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e80ba460 0x7fc5e814b560 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.776+0000 7fc5ee200640 1 -- 172.21.15.107:0/3317254996 >> 172.21.15.107:0/3317254996 conn(0x7fc5e806d3d0 msgr2=0x7fc5e80c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.776+0000 7fc5ee200640 1 -- 172.21.15.107:0/3317254996 shutdown_connections 2024-04-15T15:18:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.776+0000 7fc5ee200640 1 -- 172.21.15.107:0/3317254996 wait complete. 2024-04-15T15:18:28.789 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:28.788+0000 7f3748c11040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:18:28.794 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 5229bd2f-7a59-43df-b354-92a1648d46a9 7 2024-04-15T15:18:28.842 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:28.840+0000 7f3748c11040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-15T15:18:28.914 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:28.912+0000 7f1eb8238040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-15T15:18:28.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.984+0000 7f12ec600640 1 Processor -- start 2024-04-15T15:18:28.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.984+0000 7f12ec600640 1 -- start start 2024-04-15T15:18:28.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ec600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f12e40b3720 0x7f12e40b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ec600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3c70 0x7f12e40b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ec600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e40ba4d0 con 0x7f12e415ecc0 2024-04-15T15:18:28.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ec600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f12e415f0a0 con 0x7f12e40b3720 2024-04-15T15:18:28.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ec600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f12e40ba280 con 0x7f12e40b3c70 2024-04-15T15:18:28.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12e9200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f12e40b3720 0x7f12e40b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ea600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3c70 0x7f12e40b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12e9200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f12e40b3720 0x7f12e40b95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41842/0 (socket says 172.21.15.107:41842) 2024-04-15T15:18:28.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ea600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3c70 0x7f12e40b9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:56320/0 (socket says 172.21.15.107:56320) 2024-04-15T15:18:28.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12e9200640 1 -- 172.21.15.107:0/2434039515 learned_addr learned my addr 172.21.15.107:0/2434039515 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:28.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12e9c00640 1 --1- 172.21.15.107:0/2434039515 >> v1:172.21.15.107:6789/0 conn(0x7f12e415ecc0 0x7f12e40d2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:59914/0 (socket says 172.21.15.107:59914) 2024-04-15T15:18:28.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ea600640 1 -- 172.21.15.107:0/2434039515 >> v2:172.21.15.107:3300/0 conn(0x7f12e40b3720 msgr2=0x7f12e40b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:28.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ea600640 1 --2- 172.21.15.107:0/2434039515 >> v2:172.21.15.107:3300/0 conn(0x7f12e40b3720 0x7f12e40b95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ea600640 1 -- 172.21.15.107:0/2434039515 >> v1:172.21.15.107:6789/0 conn(0x7f12e415ecc0 legacy=0x7f12e40d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.986+0000 7f12ea600640 1 -- 172.21.15.107:0/2434039515 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e40ba7b0 con 0x7f12e40b3c70 2024-04-15T15:18:28.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.987+0000 7f12ea600640 1 --2- 172.21.15.107:0/2434039515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3c70 0x7f12e40b9b30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009d90 tx=0x7f12dc02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6cca0934d5551f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:28.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.988+0000 7f12e3e00640 1 -- 172.21.15.107:0/2434039515 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc039070 con 0x7f12e40b3c70 2024-04-15T15:18:28.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.988+0000 7f12e3e00640 1 -- 172.21.15.107:0/2434039515 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12dc02cc50 con 0x7f12e40b3c70 2024-04-15T15:18:28.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.988+0000 7f12e3e00640 1 -- 172.21.15.107:0/2434039515 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc034440 con 0x7f12e40b3c70 2024-04-15T15:18:28.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.988+0000 7f12ec600640 1 -- 172.21.15.107:0/2434039515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3c70 msgr2=0x7f12e40b9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.988+0000 7f12ec600640 1 --2- 172.21.15.107:0/2434039515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3c70 0x7f12e40b9b30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009d90 tx=0x7f12dc02bea0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.988+0000 7f12ec600640 1 -- 172.21.15.107:0/2434039515 shutdown_connections 2024-04-15T15:18:28.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.988+0000 7f12ec600640 1 --2- 172.21.15.107:0/2434039515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3c70 0x7f12e40b9b30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.988+0000 7f12ec600640 1 --2- 172.21.15.107:0/2434039515 >> v2:172.21.15.107:3300/0 conn(0x7f12e40b3720 0x7f12e40b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.988+0000 7f12ec600640 1 -- 172.21.15.107:0/2434039515 >> 172.21.15.107:0/2434039515 conn(0x7f12e4073480 msgr2=0x7f12e4073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:28.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.989+0000 7f12ec600640 1 -- 172.21.15.107:0/2434039515 shutdown_connections 2024-04-15T15:18:28.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.989+0000 7f12ec600640 1 -- 172.21.15.107:0/2434039515 wait complete. 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.991+0000 7f12ec600640 1 Processor -- start 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.991+0000 7f12ec600640 1 -- start start 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12ec600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3720 0x7f12e4160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12ec600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e40b3c70 0x7f12e40d47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12ec600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e415ecc0 0x7f12e40d4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12ec600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12e40b8aa0 con 0x7f12e40b3c70 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12ec600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12e40b90d0 con 0x7f12e415ecc0 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12ec600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f12e40d3440 con 0x7f12e40b3720 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12e9c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3720 0x7f12e4160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12ea600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e415ecc0 0x7f12e40d4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12e9c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3720 0x7f12e4160870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:56332/0 (socket says 172.21.15.107:56332) 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12e9c00640 1 -- 172.21.15.107:0/4213368941 learned_addr learned my addr 172.21.15.107:0/4213368941 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12e9200640 1 --2- 172.21.15.107:0/4213368941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e40b3c70 0x7f12e40d47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12e9c00640 1 -- 172.21.15.107:0/4213368941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e415ecc0 msgr2=0x7f12e40d4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12e9c00640 1 --2- 172.21.15.107:0/4213368941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e415ecc0 0x7f12e40d4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12e9c00640 1 -- 172.21.15.107:0/4213368941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e40b3c70 msgr2=0x7f12e40d47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12e9c00640 1 --2- 172.21.15.107:0/4213368941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e40b3c70 0x7f12e40d47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.993+0000 7f12e9c00640 1 -- 172.21.15.107:0/4213368941 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e40c8790 con 0x7f12e40b3720 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.994+0000 7f12e9c00640 1 --2- 172.21.15.107:0/4213368941 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3720 0x7f12e4160870 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c3f0 tx=0x7f12d800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.994+0000 7f12e1600640 1 -- 172.21.15.107:0/4213368941 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d80074e0 con 0x7f12e40b3720 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.994+0000 7f12e1600640 1 -- 172.21.15.107:0/4213368941 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d8007680 con 0x7f12e40b3720 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.994+0000 7f12e1600640 1 -- 172.21.15.107:0/4213368941 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8004a00 con 0x7f12e40b3720 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.994+0000 7f12ec600640 1 -- 172.21.15.107:0/4213368941 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e4154700 con 0x7f12e40b3720 2024-04-15T15:18:28.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.994+0000 7f12ec600640 1 -- 172.21.15.107:0/4213368941 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e4154cc0 con 0x7f12e40b3720 2024-04-15T15:18:28.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.994+0000 7f12e1600640 1 -- 172.21.15.107:0/4213368941 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f12d801a020 con 0x7f12e40b3720 2024-04-15T15:18:28.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.994+0000 7f12e1600640 1 -- 172.21.15.107:0/4213368941 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f12d8012a50 con 0x7f12e40b3720 2024-04-15T15:18:28.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.995+0000 7f12ec600640 1 -- 172.21.15.107:0/4213368941 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e4068c80 con 0x7f12e40b3720 2024-04-15T15:18:28.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:28.997+0000 7f12e1600640 1 -- 172.21.15.107:0/4213368941 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f12d8021070 con 0x7f12e40b3720 2024-04-15T15:18:29.028 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:29.026+0000 7f3748c11040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-15T15:18:29.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.076+0000 7f12ec600640 1 -- 172.21.15.107:0/4213368941 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5229bd2f-7a59-43df-b354-92a1648d46a9", "id": 7} v 0) -- 0x7f12e415cba0 con 0x7f12e40b3720 2024-04-15T15:18:29.083 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:29.082+0000 7f1eb8238040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-15T15:18:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.085+0000 7f12e1600640 1 -- 172.21.15.107:0/4213368941 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5229bd2f-7a59-43df-b354-92a1648d46a9", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f12d8012d70 con 0x7f12e40b3720 2024-04-15T15:18:29.086 INFO:teuthology.orchestra.run.smithi107.stdout:7 2024-04-15T15:18:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.086+0000 7f12ec600640 1 -- 172.21.15.107:0/4213368941 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3720 msgr2=0x7f12e4160870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.086+0000 7f12ec600640 1 --2- 172.21.15.107:0/4213368941 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3720 0x7f12e4160870 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c3f0 tx=0x7f12d800c8b0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.086+0000 7f12ec600640 1 -- 172.21.15.107:0/4213368941 shutdown_connections 2024-04-15T15:18:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.086+0000 7f12ec600640 1 --2- 172.21.15.107:0/4213368941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e415ecc0 0x7f12e40d4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.086+0000 7f12ec600640 1 --2- 172.21.15.107:0/4213368941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e40b3c70 0x7f12e40d47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.086+0000 7f12ec600640 1 --2- 172.21.15.107:0/4213368941 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e40b3720 0x7f12e4160870 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.086+0000 7f12ec600640 1 -- 172.21.15.107:0/4213368941 >> 172.21.15.107:0/4213368941 conn(0x7f12e4073480 msgr2=0x7f12e415d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.086+0000 7f12ec600640 1 -- 172.21.15.107:0/4213368941 shutdown_connections 2024-04-15T15:18:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:18:29.086+0000 7f12ec600640 1 -- 172.21.15.107:0/4213368941 wait complete. 2024-04-15T15:18:29.105 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-15T15:18:29.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-15T15:18:29.113 INFO:tasks.ceph.osd.0:Started 2024-04-15T15:18:29.113 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-15T15:18:29.113 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-15T15:18:29.123 INFO:tasks.ceph.osd.1:Started 2024-04-15T15:18:29.123 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-15T15:18:29.123 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-15T15:18:29.129 INFO:tasks.ceph.osd.2:Started 2024-04-15T15:18:29.129 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-15T15:18:29.129 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-15T15:18:29.134 INFO:tasks.ceph.osd.3:Started 2024-04-15T15:18:29.134 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-15T15:18:29.134 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-15T15:18:29.140 INFO:tasks.ceph.osd.4:Started 2024-04-15T15:18:29.140 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-15T15:18:29.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-15T15:18:29.146 INFO:tasks.ceph.osd.5:Started 2024-04-15T15:18:29.146 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-15T15:18:29.146 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-15T15:18:29.152 INFO:tasks.ceph.osd.6:Started 2024-04-15T15:18:29.152 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-15T15:18:29.152 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-15T15:18:29.157 INFO:tasks.ceph.osd.7:Started 2024-04-15T15:18:29.158 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:29.254 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:29.252+0000 7f1eb8238040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-15T15:18:29.330 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:29.329+0000 7f1eb8238040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-15T15:18:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.373+0000 7f0e91a00640 1 Processor -- start 2024-04-15T15:18:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.373+0000 7f0e91a00640 1 -- start start 2024-04-15T15:18:29.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.374+0000 7f0e91a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c0d3e20 0x7f0e8c0d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:29.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.374+0000 7f0e91a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e8c15ee00 0x7f0e8c0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:29.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.374+0000 7f0e91a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0e8c0cf6f0 0x7f0e8c0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:29.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.374+0000 7f0e91a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f0e8c0cfba0 con 0x7f0e8c15ee00 2024-04-15T15:18:29.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.374+0000 7f0e91a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f0e8c0d2b10 con 0x7f0e8c0cf6f0 2024-04-15T15:18:29.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.374+0000 7f0e91a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8c0d33b0 con 0x7f0e8c0d3e20 2024-04-15T15:18:29.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.375+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c0d3e20 0x7f0e8c0d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.375+0000 7f0e8a000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e8c15ee00 0x7f0e8c0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.375+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c0d3e20 0x7f0e8c0d4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46686/0 (socket says 172.21.15.49:46686) 2024-04-15T15:18:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.375+0000 7f0e8aa00640 1 -- 172.21.15.49:0/564698684 learned_addr learned my addr 172.21.15.49:0/564698684 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.375+0000 7f0e8b400640 1 --2- 172.21.15.49:0/564698684 >> v2:172.21.15.107:3300/0 conn(0x7f0e8c0cf6f0 0x7f0e8c0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.375+0000 7f0e8aa00640 1 -- 172.21.15.49:0/564698684 >> v2:172.21.15.107:3300/0 conn(0x7f0e8c0cf6f0 msgr2=0x7f0e8c0b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.375+0000 7f0e8aa00640 1 --2- 172.21.15.49:0/564698684 >> v2:172.21.15.107:3300/0 conn(0x7f0e8c0cf6f0 0x7f0e8c0b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.375+0000 7f0e8aa00640 1 -- 172.21.15.49:0/564698684 >> v2:172.21.15.49:3300/0 conn(0x7f0e8c15ee00 msgr2=0x7f0e8c0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.375+0000 7f0e8aa00640 1 --2- 172.21.15.49:0/564698684 >> v2:172.21.15.49:3300/0 conn(0x7f0e8c15ee00 0x7f0e8c0cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.375+0000 7f0e8aa00640 1 -- 172.21.15.49:0/564698684 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c0d4740 con 0x7f0e8c0d3e20 2024-04-15T15:18:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.376+0000 7f0e8aa00640 1 --2- 172.21.15.49:0/564698684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c0d3e20 0x7f0e8c0d4200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c0b74f0 tx=0x7f0e7c02b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c209dfc9246e31c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e89600640 1 -- 172.21.15.49:0/564698684 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c034070 con 0x7f0e8c0d3e20 2024-04-15T15:18:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e89600640 1 -- 172.21.15.49:0/564698684 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e7c030860 con 0x7f0e8c0d3e20 2024-04-15T15:18:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e89600640 1 -- 172.21.15.49:0/564698684 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c030c30 con 0x7f0e8c0d3e20 2024-04-15T15:18:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e91a00640 1 -- 172.21.15.49:0/564698684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c0d3e20 msgr2=0x7f0e8c0d4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e91a00640 1 --2- 172.21.15.49:0/564698684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c0d3e20 0x7f0e8c0d4200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c0b74f0 tx=0x7f0e7c02b9a0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e91a00640 1 -- 172.21.15.49:0/564698684 shutdown_connections 2024-04-15T15:18:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e91a00640 1 --2- 172.21.15.49:0/564698684 >> v2:172.21.15.107:3300/0 conn(0x7f0e8c0cf6f0 0x7f0e8c0b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e91a00640 1 --2- 172.21.15.49:0/564698684 >> v2:172.21.15.49:3300/0 conn(0x7f0e8c15ee00 0x7f0e8c0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e91a00640 1 --2- 172.21.15.49:0/564698684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c0d3e20 0x7f0e8c0d4200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e91a00640 1 -- 172.21.15.49:0/564698684 >> 172.21.15.49:0/564698684 conn(0x7f0e8c06d470 msgr2=0x7f0e8c06d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e91a00640 1 -- 172.21.15.49:0/564698684 shutdown_connections 2024-04-15T15:18:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.377+0000 7f0e91a00640 1 -- 172.21.15.49:0/564698684 wait complete. 2024-04-15T15:18:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.379+0000 7f0e91a00640 1 Processor -- start 2024-04-15T15:18:29.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.379+0000 7f0e91a00640 1 -- start start 2024-04-15T15:18:29.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e91a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cf6f0 0x7f0e8c1fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:29.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e91a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e8c0d3e20 0x7f0e8c0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:29.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e91a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c15ee00 0x7f0e8c1fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:29.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e91a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c0d3040 con 0x7f0e8c0cf6f0 2024-04-15T15:18:29.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e91a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c0d2d60 con 0x7f0e8c0d3e20 2024-04-15T15:18:29.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e91a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8c0d3840 con 0x7f0e8c15ee00 2024-04-15T15:18:29.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cf6f0 0x7f0e8c1fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cf6f0 0x7f0e8c1fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51654/0 (socket says 172.21.15.49:51654) 2024-04-15T15:18:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e8aa00640 1 -- 172.21.15.49:0/2816051007 learned_addr learned my addr 172.21.15.49:0/2816051007 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e8b400640 1 --2- 172.21.15.49:0/2816051007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c15ee00 0x7f0e8c1fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.380+0000 7f0e8a000640 1 --2- 172.21.15.49:0/2816051007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e8c0d3e20 0x7f0e8c0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e8b400640 1 -- 172.21.15.49:0/2816051007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e8c0d3e20 msgr2=0x7f0e8c0c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e8b400640 1 --2- 172.21.15.49:0/2816051007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e8c0d3e20 0x7f0e8c0c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e8b400640 1 -- 172.21.15.49:0/2816051007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cf6f0 msgr2=0x7f0e8c1fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e8b400640 1 --2- 172.21.15.49:0/2816051007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cf6f0 0x7f0e8c1fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e8b400640 1 -- 172.21.15.49:0/2816051007 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c0b4c40 con 0x7f0e8c15ee00 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e8aa00640 1 --2- 172.21.15.49:0/2816051007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cf6f0 0x7f0e8c1fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e8b400640 1 --2- 172.21.15.49:0/2816051007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c15ee00 0x7f0e8c1fc2a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400ef00 tx=0x7f0e8400b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e82a00640 1 -- 172.21.15.49:0/2816051007 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84018020 con 0x7f0e8c15ee00 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e82a00640 1 -- 172.21.15.49:0/2816051007 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e84014400 con 0x7f0e8c15ee00 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e82a00640 1 -- 172.21.15.49:0/2816051007 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84009e60 con 0x7f0e8c15ee00 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.381+0000 7f0e91a00640 1 -- 172.21.15.49:0/2816051007 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c157070 con 0x7f0e8c15ee00 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.382+0000 7f0e91a00640 1 -- 172.21.15.49:0/2816051007 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c0b6f10 con 0x7f0e8c15ee00 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.382+0000 7f0e82a00640 1 -- 172.21.15.49:0/2816051007 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e84005d10 con 0x7f0e8c15ee00 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.382+0000 7f0e82a00640 1 -- 172.21.15.49:0/2816051007 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0e8401d400 con 0x7f0e8c15ee00 2024-04-15T15:18:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.383+0000 7f0e91a00640 1 -- 172.21.15.49:0/2816051007 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8c068c80 con 0x7f0e8c15ee00 2024-04-15T15:18:29.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.385+0000 7f0e82a00640 1 -- 172.21.15.49:0/2816051007 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0e84022030 con 0x7f0e8c15ee00 2024-04-15T15:18:29.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.451+0000 7f0e91a00640 1 -- 172.21.15.49:0/2816051007 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e8c0c80b0 con 0x7f0e8c15ee00 2024-04-15T15:18:29.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.452+0000 7f0e82a00640 1 -- 172.21.15.49:0/2816051007 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0e8c0b4c40 con 0x7f0e8c15ee00 2024-04-15T15:18:29.454 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:29.454 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:29.079024+0000","last_up_change":"0.000000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:29.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.453+0000 7f0e91a00640 1 -- 172.21.15.49:0/2816051007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c15ee00 msgr2=0x7f0e8c1fc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:29.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.453+0000 7f0e91a00640 1 --2- 172.21.15.49:0/2816051007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c15ee00 0x7f0e8c1fc2a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400ef00 tx=0x7f0e8400b480 comp rx=0 tx=0).stop 2024-04-15T15:18:29.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.453+0000 7f0e91a00640 1 -- 172.21.15.49:0/2816051007 shutdown_connections 2024-04-15T15:18:29.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.453+0000 7f0e91a00640 1 --2- 172.21.15.49:0/2816051007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e8c15ee00 0x7f0e8c1fc2a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.453+0000 7f0e91a00640 1 --2- 172.21.15.49:0/2816051007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e8c0d3e20 0x7f0e8c0c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.453+0000 7f0e91a00640 1 --2- 172.21.15.49:0/2816051007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cf6f0 0x7f0e8c1fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.453+0000 7f0e91a00640 1 -- 172.21.15.49:0/2816051007 >> 172.21.15.49:0/2816051007 conn(0x7f0e8c06d470 msgr2=0x7f0e8c15fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.453+0000 7f0e91a00640 1 -- 172.21.15.49:0/2816051007 shutdown_connections 2024-04-15T15:18:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:29.453+0000 7f0e91a00640 1 -- 172.21.15.49:0/2816051007 wait complete. 2024-04-15T15:18:29.468 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:29.466+0000 7f1eb8238040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:18:29.470 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-15T15:18:29.471 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-15T15:18:29.582 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:29.580+0000 7f1eb8238040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-15T15:18:29.760 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:29.758+0000 7f1eb8238040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:18:29.810 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:29.808+0000 7f1eb8238040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-15T15:18:29.817 INFO:tasks.ceph.osd.4.smithi107.stderr:2024-04-15T15:18:29.815+0000 7f77e888b640 -1 Falling back to public interface 2024-04-15T15:18:29.853 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-04-15T15:18:29.851+0000 7fbb4d6b1640 -1 Falling back to public interface 2024-04-15T15:18:29.853 INFO:tasks.ceph.osd.6.smithi107.stderr:2024-04-15T15:18:29.851+0000 7f66b1506640 -1 Falling back to public interface 2024-04-15T15:18:29.854 INFO:tasks.ceph.osd.6.smithi107.stderr:2024-04-15T15:18:29.852+0000 7f66b0000640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-04-15T15:18:29.858 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-15T15:18:29.856+0000 7f6c99023640 -1 Falling back to public interface 2024-04-15T15:18:29.860 INFO:tasks.ceph.osd.7.smithi107.stderr:2024-04-15T15:18:29.859+0000 7f476e01e640 -1 Falling back to public interface 2024-04-15T15:18:29.861 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-15T15:18:29.860+0000 7f3d1fd73640 -1 Falling back to public interface 2024-04-15T15:18:29.866 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-15T15:18:29.865+0000 7f12578ac640 -1 Falling back to public interface 2024-04-15T15:18:29.875 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-15T15:18:29.874+0000 7fac70111640 -1 Falling back to public interface 2024-04-15T15:18:29.996 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:29.994+0000 7f1eb8238040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-15T15:18:30.207 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:30.205+0000 7f3748c11040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-15T15:18:30.278 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:30.437 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.436+0000 7f85c7600640 1 Processor -- start 2024-04-15T15:18:30.437 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.436+0000 7f85c7600640 1 -- start start 2024-04-15T15:18:30.439 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.437+0000 7f85c7600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85c015ecd0 0x7f85c00d2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:30.439 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.437+0000 7f85c7600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f85c00ba370 0x7f85c00cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:30.439 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.437+0000 7f85c7600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f85c00cfc80 0x7f85c006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:30.439 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.437+0000 7f85c7600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f85c015f0b0 con 0x7f85c00ba370 2024-04-15T15:18:30.439 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.437+0000 7f85c7600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f85c00d4210 con 0x7f85c00cfc80 2024-04-15T15:18:30.440 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.437+0000 7f85c7600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f85c00d3600 con 0x7f85c015ecd0 2024-04-15T15:18:30.440 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.438+0000 7f85bfe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f85c00ba370 0x7f85c00cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:30.440 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.438+0000 7f85bfe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f85c00ba370 0x7f85c00cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51660/0 (socket says 172.21.15.49:51660) 2024-04-15T15:18:30.440 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.438+0000 7f85bfe00640 1 -- 172.21.15.49:0/1150289601 learned_addr learned my addr 172.21.15.49:0/1150289601 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:30.440 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.438+0000 7f85c4c00640 1 --2- 172.21.15.49:0/1150289601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85c015ecd0 0x7f85c00d2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:30.440 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.438+0000 7f85c5600640 1 --2- 172.21.15.49:0/1150289601 >> v2:172.21.15.107:3300/0 conn(0x7f85c00cfc80 0x7f85c006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:30.440 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.438+0000 7f85bfe00640 1 -- 172.21.15.49:0/1150289601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85c015ecd0 msgr2=0x7f85c00d2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:30.441 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.438+0000 7f85bfe00640 1 --2- 172.21.15.49:0/1150289601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85c015ecd0 0x7f85c00d2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.441 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.438+0000 7f85bfe00640 1 -- 172.21.15.49:0/1150289601 >> v2:172.21.15.107:3300/0 conn(0x7f85c00cfc80 msgr2=0x7f85c006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:30.441 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.438+0000 7f85bfe00640 1 --2- 172.21.15.49:0/1150289601 >> v2:172.21.15.107:3300/0 conn(0x7f85c00cfc80 0x7f85c006cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.441 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.438+0000 7f85bfe00640 1 -- 172.21.15.49:0/1150289601 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c00d38e0 con 0x7f85c00ba370 2024-04-15T15:18:30.441 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.439+0000 7f85bfe00640 1 --2- 172.21.15.49:0/1150289601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00ba370 0x7f85c00cf740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f85c006dfc0 tx=0x7f85b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdff4a2c0ec40d0a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:30.442 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.440+0000 7f85bf400640 1 -- 172.21.15.49:0/1150289601 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0035020 con 0x7f85c00ba370 2024-04-15T15:18:30.442 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.440+0000 7f85bf400640 1 -- 172.21.15.49:0/1150289601 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b0031870 con 0x7f85c00ba370 2024-04-15T15:18:30.443 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.440+0000 7f85bf400640 1 -- 172.21.15.49:0/1150289601 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0031c40 con 0x7f85c00ba370 2024-04-15T15:18:30.443 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.440+0000 7f85c7600640 1 -- 172.21.15.49:0/1150289601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00ba370 msgr2=0x7f85c00cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:30.443 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.440+0000 7f85c7600640 1 --2- 172.21.15.49:0/1150289601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00ba370 0x7f85c00cf740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f85c006dfc0 tx=0x7f85b002b910 comp rx=0 tx=0).stop 2024-04-15T15:18:30.443 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.440+0000 7f85c7600640 1 -- 172.21.15.49:0/1150289601 shutdown_connections 2024-04-15T15:18:30.443 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.440+0000 7f85c7600640 1 --2- 172.21.15.49:0/1150289601 >> v2:172.21.15.107:3300/0 conn(0x7f85c00cfc80 0x7f85c006cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.443 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.440+0000 7f85c7600640 1 --2- 172.21.15.49:0/1150289601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00ba370 0x7f85c00cf740 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.443 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.440+0000 7f85c7600640 1 --2- 172.21.15.49:0/1150289601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85c015ecd0 0x7f85c00d2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.443 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.440+0000 7f85c7600640 1 -- 172.21.15.49:0/1150289601 >> 172.21.15.49:0/1150289601 conn(0x7f85c00d1810 msgr2=0x7f85c00d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:30.443 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.441+0000 7f85c7600640 1 -- 172.21.15.49:0/1150289601 shutdown_connections 2024-04-15T15:18:30.443 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.441+0000 7f85c7600640 1 -- 172.21.15.49:0/1150289601 wait complete. 2024-04-15T15:18:30.444 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.442+0000 7f85c7600640 1 Processor -- start 2024-04-15T15:18:30.444 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.442+0000 7f85c7600640 1 -- start start 2024-04-15T15:18:30.445 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.443+0000 7f85c7600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c00ba370 0x7f85c00caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:30.445 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.443+0000 7f85c7600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00cfc80 0x7f85c01f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:30.445 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.443+0000 7f85c7600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85c015ecd0 0x7f85c01f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.443+0000 7f85c7600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85c00d46a0 con 0x7f85c00cfc80 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.443+0000 7f85c7600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85c00d3ea0 con 0x7f85c00ba370 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.443+0000 7f85c7600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f85c00d3260 con 0x7f85c015ecd0 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85bfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00cfc80 0x7f85c01f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85bfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00cfc80 0x7f85c01f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51674/0 (socket says 172.21.15.49:51674) 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85bfe00640 1 -- 172.21.15.49:0/530689137 learned_addr learned my addr 172.21.15.49:0/530689137 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85c5600640 1 --2- 172.21.15.49:0/530689137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85c015ecd0 0x7f85c01f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85c4c00640 1 --2- 172.21.15.49:0/530689137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c00ba370 0x7f85c00caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85bfe00640 1 -- 172.21.15.49:0/530689137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85c015ecd0 msgr2=0x7f85c01f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85bfe00640 1 --2- 172.21.15.49:0/530689137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85c015ecd0 0x7f85c01f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85bfe00640 1 -- 172.21.15.49:0/530689137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c00ba370 msgr2=0x7f85c00caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:30.446 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85bfe00640 1 --2- 172.21.15.49:0/530689137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c00ba370 0x7f85c00caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.447 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85bfe00640 1 -- 172.21.15.49:0/530689137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c01f58d0 con 0x7f85c00cfc80 2024-04-15T15:18:30.448 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.444+0000 7f85bfe00640 1 --2- 172.21.15.49:0/530689137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00cfc80 0x7f85c01f5390 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f85b0000c00 tx=0x7f85b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:30.448 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.445+0000 7f85bcc00640 1 -- 172.21.15.49:0/530689137 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0040070 con 0x7f85c00cfc80 2024-04-15T15:18:30.448 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.445+0000 7f85bcc00640 1 -- 172.21.15.49:0/530689137 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b0031e10 con 0x7f85c00cfc80 2024-04-15T15:18:30.448 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.445+0000 7f85bcc00640 1 -- 172.21.15.49:0/530689137 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0035040 con 0x7f85c00cfc80 2024-04-15T15:18:30.448 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.445+0000 7f85c7600640 1 -- 172.21.15.49:0/530689137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c01613f0 con 0x7f85c00cfc80 2024-04-15T15:18:30.448 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.445+0000 7f85c7600640 1 -- 172.21.15.49:0/530689137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c01619b0 con 0x7f85c00cfc80 2024-04-15T15:18:30.448 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.445+0000 7f85bcc00640 1 -- 172.21.15.49:0/530689137 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f85b004b060 con 0x7f85c00cfc80 2024-04-15T15:18:30.448 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.445+0000 7f85bcc00640 1 -- 172.21.15.49:0/530689137 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f85b003ab40 con 0x7f85c00cfc80 2024-04-15T15:18:30.448 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.446+0000 7f85c7600640 1 -- 172.21.15.49:0/530689137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c006e0c0 con 0x7f85c00cfc80 2024-04-15T15:18:30.450 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.448+0000 7f85bcc00640 1 -- 172.21.15.49:0/530689137 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f85c01f58d0 con 0x7f85c00cfc80 2024-04-15T15:18:30.519 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.517+0000 7f85c7600640 1 -- 172.21.15.49:0/530689137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85c00b9050 con 0x7f85c00cfc80 2024-04-15T15:18:30.519 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.518+0000 7f85bcc00640 1 -- 172.21.15.49:0/530689137 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f85b003a470 con 0x7f85c00cfc80 2024-04-15T15:18:30.520 INFO:teuthology.misc.health.smithi049.stdout: 2024-04-15T15:18:30.520 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":10,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:29.079024+0000","last_up_change":"0.000000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:30.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.519+0000 7f85c7600640 1 -- 172.21.15.49:0/530689137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00cfc80 msgr2=0x7f85c01f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:30.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.519+0000 7f85c7600640 1 --2- 172.21.15.49:0/530689137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00cfc80 0x7f85c01f5390 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f85b0000c00 tx=0x7f85b00046c0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.519+0000 7f85c7600640 1 -- 172.21.15.49:0/530689137 shutdown_connections 2024-04-15T15:18:30.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.519+0000 7f85c7600640 1 --2- 172.21.15.49:0/530689137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85c015ecd0 0x7f85c01f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.519+0000 7f85c7600640 1 --2- 172.21.15.49:0/530689137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85c00cfc80 0x7f85c01f5390 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.519+0000 7f85c7600640 1 --2- 172.21.15.49:0/530689137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c00ba370 0x7f85c00caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:30.522 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.519+0000 7f85c7600640 1 -- 172.21.15.49:0/530689137 >> 172.21.15.49:0/530689137 conn(0x7f85c00d1810 msgr2=0x7f85c015db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:30.522 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.520+0000 7f85c7600640 1 -- 172.21.15.49:0/530689137 shutdown_connections 2024-04-15T15:18:30.522 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:30.520+0000 7f85c7600640 1 -- 172.21.15.49:0/530689137 wait complete. 2024-04-15T15:18:30.534 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-15T15:18:30.548 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:30.546+0000 7f3748c11040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-15T15:18:30.612 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:30.610+0000 7f3748c11040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-15T15:18:30.752 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:30.750+0000 7f3748c11040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-15T15:18:31.036 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:31.034+0000 7f1eb8238040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-15T15:18:31.351 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:31.349+0000 7f1eb8238040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-15T15:18:31.410 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:31.408+0000 7f1eb8238040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-15T15:18:31.543 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-15T15:18:31.542+0000 7f1eb8238040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-15T15:18:31.793 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:31.791+0000 7f36e9200640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-15T15:18:31.795 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:18:31.793+0000 7f36e9200640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-15T15:18:33.227 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-15T15:18:33.225+0000 7f3d1fd73640 -1 osd.0 0 log_to_monitors true 2024-04-15T15:18:33.270 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-15T15:18:33.268+0000 7f12578ac640 -1 osd.2 0 log_to_monitors true 2024-04-15T15:18:33.449 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-15T15:18:33.447+0000 7f6c99023640 -1 osd.1 0 log_to_monitors true 2024-04-15T15:18:33.466 INFO:tasks.ceph.osd.6.smithi107.stderr:2024-04-15T15:18:33.464+0000 7f66b1506640 -1 osd.6 0 log_to_monitors true 2024-04-15T15:18:33.482 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-15T15:18:33.480+0000 7fac70111640 -1 osd.3 0 log_to_monitors true 2024-04-15T15:18:33.502 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-04-15T15:18:33.501+0000 7fbb4d6b1640 -1 osd.5 0 log_to_monitors true 2024-04-15T15:18:33.524 INFO:tasks.ceph.osd.7.smithi107.stderr:2024-04-15T15:18:33.522+0000 7f476e01e640 -1 osd.7 0 log_to_monitors true 2024-04-15T15:18:33.646 INFO:tasks.ceph.osd.4.smithi107.stderr:2024-04-15T15:18:33.644+0000 7f77e888b640 -1 osd.4 0 log_to_monitors true 2024-04-15T15:18:36.361 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-15T15:18:36.358+0000 7f3d1ac00640 -1 osd.0 0 waiting for initial osdmap 2024-04-15T15:18:36.361 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-15T15:18:36.359+0000 7f6c94200640 -1 osd.1 0 waiting for initial osdmap 2024-04-15T15:18:36.365 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-15T15:18:36.362+0000 7fac6b000640 -1 osd.3 0 waiting for initial osdmap 2024-04-15T15:18:36.371 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-15T15:18:36.369+0000 7f6c89800640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:18:36.372 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-15T15:18:36.369+0000 7fac61000640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:18:36.373 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-15T15:18:36.370+0000 7f3d10c00640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:18:36.388 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-15T15:18:36.386+0000 7f1253200640 -1 osd.2 0 waiting for initial osdmap 2024-04-15T15:18:36.397 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-15T15:18:36.395+0000 7f1248800640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:18:36.630 INFO:tasks.ceph.osd.6.smithi107.stderr:2024-04-15T15:18:36.629+0000 7f66ac400640 -1 osd.6 0 waiting for initial osdmap 2024-04-15T15:18:36.637 INFO:tasks.ceph.osd.7.smithi107.stderr:2024-04-15T15:18:36.633+0000 7f4769200640 -1 osd.7 0 waiting for initial osdmap 2024-04-15T15:18:36.637 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-04-15T15:18:36.634+0000 7fbb48600640 -1 osd.5 0 waiting for initial osdmap 2024-04-15T15:18:36.640 INFO:tasks.ceph.osd.6.smithi107.stderr:2024-04-15T15:18:36.638+0000 7f66a2400640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:18:36.644 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-04-15T15:18:36.643+0000 7fbb3e600640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:18:36.651 INFO:tasks.ceph.osd.7.smithi107.stderr:2024-04-15T15:18:36.649+0000 7f475e800640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:18:37.042 INFO:tasks.ceph.osd.4.smithi107.stderr:2024-04-15T15:18:37.041+0000 7f77e4200640 -1 osd.4 0 waiting for initial osdmap 2024-04-15T15:18:37.050 INFO:tasks.ceph.osd.4.smithi107.stderr:2024-04-15T15:18:37.049+0000 7f77d9800640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:18:37.174 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:18:37.183 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:18:37.340 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:37.519 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.517+0000 7f2505400640 1 Processor -- start 2024-04-15T15:18:37.520 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.517+0000 7f2505400640 1 -- start start 2024-04-15T15:18:37.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.518+0000 7f2505400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f250015f4b0 0x7f250015f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.518+0000 7f2505400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25000b3690 0x7f2500072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.518+0000 7f2505400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f25000b3b10 0x7f2500072880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.518+0000 7f2505400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f250015fdd0 con 0x7f25000b3690 2024-04-15T15:18:37.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.518+0000 7f2505400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f25001630b0 con 0x7f25000b3b10 2024-04-15T15:18:37.521 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.518+0000 7f2505400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f25001624a0 con 0x7f250015f4b0 2024-04-15T15:18:37.522 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f24f7e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25000b3690 0x7f2500072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.522 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f24f7e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25000b3690 0x7f2500072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44918/0 (socket says 172.21.15.49:44918) 2024-04-15T15:18:37.522 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f24f7e00640 1 -- 172.21.15.49:0/2069157907 learned_addr learned my addr 172.21.15.49:0/2069157907 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:37.523 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f24f7e00640 1 -- 172.21.15.49:0/2069157907 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f250015f4b0 msgr2=0x7f250015f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:37.523 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f24ffe00640 1 --2- 172.21.15.49:0/2069157907 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f250015f4b0 0x7f250015f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.523 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f2504a00640 1 --2- 172.21.15.49:0/2069157907 >> v2:172.21.15.107:3300/0 conn(0x7f25000b3b10 0x7f2500072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.523 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f24f7e00640 1 --2- 172.21.15.49:0/2069157907 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f250015f4b0 0x7f250015f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.523 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f24f7e00640 1 -- 172.21.15.49:0/2069157907 >> v2:172.21.15.107:3300/0 conn(0x7f25000b3b10 msgr2=0x7f2500072880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:37.523 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f24f7e00640 1 --2- 172.21.15.49:0/2069157907 >> v2:172.21.15.107:3300/0 conn(0x7f25000b3b10 0x7f2500072880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.523 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f24f7e00640 1 -- 172.21.15.49:0/2069157907 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2500162780 con 0x7f25000b3690 2024-04-15T15:18:37.523 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.519+0000 7f2504a00640 1 --2- 172.21.15.49:0/2069157907 >> v2:172.21.15.107:3300/0 conn(0x7f25000b3b10 0x7f2500072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:37.523 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.520+0000 7f24f7e00640 1 --2- 172.21.15.49:0/2069157907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25000b3690 0x7f2500072340 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2500159480 tx=0x7f24e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248d3de657813f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:37.524 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f24ff400640 1 -- 172.21.15.49:0/2069157907 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8035020 con 0x7f25000b3690 2024-04-15T15:18:37.524 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f24ff400640 1 -- 172.21.15.49:0/2069157907 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e8031870 con 0x7f25000b3690 2024-04-15T15:18:37.524 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f24ff400640 1 -- 172.21.15.49:0/2069157907 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8031c40 con 0x7f25000b3690 2024-04-15T15:18:37.525 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f2505400640 1 -- 172.21.15.49:0/2069157907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25000b3690 msgr2=0x7f2500072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:37.525 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f2505400640 1 --2- 172.21.15.49:0/2069157907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25000b3690 0x7f2500072340 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2500159480 tx=0x7f24e802b910 comp rx=0 tx=0).stop 2024-04-15T15:18:37.525 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f2505400640 1 -- 172.21.15.49:0/2069157907 shutdown_connections 2024-04-15T15:18:37.525 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f2505400640 1 --2- 172.21.15.49:0/2069157907 >> v2:172.21.15.107:3300/0 conn(0x7f25000b3b10 0x7f2500072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.525 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f2505400640 1 --2- 172.21.15.49:0/2069157907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25000b3690 0x7f2500072340 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.525 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f2505400640 1 --2- 172.21.15.49:0/2069157907 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f250015f4b0 0x7f250015f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.525 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f2505400640 1 -- 172.21.15.49:0/2069157907 >> 172.21.15.49:0/2069157907 conn(0x7f250006bd30 msgr2=0x7f250006c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:37.525 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f2505400640 1 -- 172.21.15.49:0/2069157907 shutdown_connections 2024-04-15T15:18:37.525 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.521+0000 7f2505400640 1 -- 172.21.15.49:0/2069157907 wait complete. 2024-04-15T15:18:37.526 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.523+0000 7f2505400640 1 Processor -- start 2024-04-15T15:18:37.526 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.523+0000 7f2505400640 1 -- start start 2024-04-15T15:18:37.527 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.524+0000 7f2505400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25000b3690 0x7f250014cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.527 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.524+0000 7f2505400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25000b3b10 0x7f2500156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.527 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.524+0000 7f2505400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f250015f4b0 0x7f2500157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.524+0000 7f2505400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f250006e000 con 0x7f250015f4b0 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.524+0000 7f2505400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2500162d40 con 0x7f25000b3b10 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.524+0000 7f2505400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f250006e180 con 0x7f25000b3690 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.524+0000 7f2504a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f250015f4b0 0x7f2500157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.524+0000 7f2504a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f250015f4b0 0x7f2500157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44932/0 (socket says 172.21.15.49:44932) 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.524+0000 7f2504a00640 1 -- 172.21.15.49:0/3129965290 learned_addr learned my addr 172.21.15.49:0/3129965290 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f24ffe00640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25000b3690 0x7f250014cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f24f7e00640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25000b3b10 0x7f2500156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f2504a00640 1 -- 172.21.15.49:0/3129965290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25000b3690 msgr2=0x7f250014cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f2504a00640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25000b3690 0x7f250014cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f2504a00640 1 -- 172.21.15.49:0/3129965290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25000b3b10 msgr2=0x7f2500156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f2504a00640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25000b3b10 0x7f2500156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.528 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f2504a00640 1 -- 172.21.15.49:0/3129965290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25000c5600 con 0x7f250015f4b0 2024-04-15T15:18:37.529 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f2504a00640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f250015f4b0 0x7f2500157930 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24f800cef0 tx=0x7f24f800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:37.529 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f24fcc00640 1 -- 172.21.15.49:0/3129965290 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f801a070 con 0x7f250015f4b0 2024-04-15T15:18:37.529 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f24fcc00640 1 -- 172.21.15.49:0/3129965290 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f800d840 con 0x7f250015f4b0 2024-04-15T15:18:37.529 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f2505400640 1 -- 172.21.15.49:0/3129965290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25000c58e0 con 0x7f250015f4b0 2024-04-15T15:18:37.529 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.525+0000 7f24fcc00640 1 -- 172.21.15.49:0/3129965290 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f800dc30 con 0x7f250015f4b0 2024-04-15T15:18:37.530 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.526+0000 7f2505400640 1 -- 172.21.15.49:0/3129965290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25000c5ea0 con 0x7f250015f4b0 2024-04-15T15:18:37.530 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.527+0000 7f2505400640 1 -- 172.21.15.49:0/3129965290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25000b4e60 con 0x7f250015f4b0 2024-04-15T15:18:37.530 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.528+0000 7f24fcc00640 1 -- 172.21.15.49:0/3129965290 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f24f801d070 con 0x7f250015f4b0 2024-04-15T15:18:37.532 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.529+0000 7f24fcc00640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24dc06c860 0x7f24dc06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.532 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.529+0000 7f24fcc00640 1 -- 172.21.15.49:0/3129965290 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4471+0+0 (secure 0 0 0) 0x7f24f808bd80 con 0x7f250015f4b0 2024-04-15T15:18:37.532 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.529+0000 7f24ffe00640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24dc06c860 0x7f24dc06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.532 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.530+0000 7f24ffe00640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24dc06c860 0x7f24dc06ed20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f24f000b7c0 tx=0x7f24f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:37.537 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.535+0000 7f24fcc00640 1 -- 172.21.15.49:0/3129965290 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25000b4e60 con 0x7f250015f4b0 2024-04-15T15:18:37.745 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.743+0000 7f2505400640 1 -- 172.21.15.49:0/3129965290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25000c7820 con 0x7f250015f4b0 2024-04-15T15:18:37.746 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.744+0000 7f24fcc00640 1 -- 172.21.15.49:0/3129965290 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10909 (secure 0 0 0) 0x7f24f805a760 con 0x7f250015f4b0 2024-04-15T15:18:37.747 INFO:teuthology.misc.health.smithi049.stdout: 2024-04-15T15:18:37.747 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":14,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:37.367735+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:37.750 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.746+0000 7f2505400640 1 -- 172.21.15.49:0/3129965290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24dc06c860 msgr2=0x7f24dc06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:37.750 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.746+0000 7f2505400640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24dc06c860 0x7f24dc06ed20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f24f000b7c0 tx=0x7f24f0008040 comp rx=0 tx=0).stop 2024-04-15T15:18:37.750 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.746+0000 7f2505400640 1 -- 172.21.15.49:0/3129965290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f250015f4b0 msgr2=0x7f2500157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:37.750 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.746+0000 7f2505400640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f250015f4b0 0x7f2500157930 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24f800cef0 tx=0x7f24f800a710 comp rx=0 tx=0).stop 2024-04-15T15:18:37.750 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.747+0000 7f2505400640 1 -- 172.21.15.49:0/3129965290 shutdown_connections 2024-04-15T15:18:37.750 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.747+0000 7f2505400640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24dc06c860 0x7f24dc06ed20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.751 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.747+0000 7f2505400640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f250015f4b0 0x7f2500157930 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.751 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.747+0000 7f2505400640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25000b3b10 0x7f2500156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.751 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.747+0000 7f2505400640 1 --2- 172.21.15.49:0/3129965290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25000b3690 0x7f250014cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.751 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.747+0000 7f2505400640 1 -- 172.21.15.49:0/3129965290 >> 172.21.15.49:0/3129965290 conn(0x7f250006bd30 msgr2=0x7f25000b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:37.751 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.747+0000 7f2505400640 1 -- 172.21.15.49:0/3129965290 shutdown_connections 2024-04-15T15:18:37.751 INFO:teuthology.misc.health.smithi049.stderr:2024-04-15T15:18:37.747+0000 7f2505400640 1 -- 172.21.15.49:0/3129965290 wait complete. 2024-04-15T15:18:37.765 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-15T15:18:37.766 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-15T15:18:37.766 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-15T15:18:37.766 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-15T15:18:37.773 INFO:tasks.ceph.mds.a:Started 2024-04-15T15:18:37.773 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-15T15:18:37.773 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-15T15:18:37.781 INFO:tasks.ceph.mds.c:Started 2024-04-15T15:18:37.782 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-15T15:18:37.782 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-15T15:18:37.790 INFO:tasks.ceph.mds.b:Started 2024-04-15T15:18:37.790 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-15T15:18:37.790 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-15T15:18:37.797 INFO:tasks.ceph.mds.d:Started 2024-04-15T15:18:37.797 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-15T15:18:37.798 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:37.932 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2024-04-15T15:18:37.934 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2024-04-15T15:18:37.952 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-15T15:18:37.955 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-15T15:18:37.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.987+0000 7f7acb000640 1 Processor -- start 2024-04-15T15:18:37.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.987+0000 7f7acb000640 1 -- start start 2024-04-15T15:18:37.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.988+0000 7f7acb000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7ac4158f20 0x7f7ac4160020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.988+0000 7f7acb000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 0x7f7ac40d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.988+0000 7f7acb000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac4160560 con 0x7f7ac40738f0 2024-04-15T15:18:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.988+0000 7f7acb000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7ac4159300 con 0x7f7ac4158f20 2024-04-15T15:18:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.988+0000 7f7acb000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac40cf600 con 0x7f7ac40b39a0 2024-04-15T15:18:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.988+0000 7f7ac9c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7ac4158f20 0x7f7ac4160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.989+0000 7f7ac9c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7ac4158f20 0x7f7ac4160020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44964/0 (socket says 172.21.15.49:44964) 2024-04-15T15:18:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.989+0000 7f7ac9c00640 1 -- 172.21.15.49:0/2961557375 learned_addr learned my addr 172.21.15.49:0/2961557375 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.989+0000 7f7ac9200640 1 --1- 172.21.15.49:0/2961557375 >> v1:172.21.15.49:6789/0 conn(0x7f7ac40738f0 0x7f7ac40b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48794/0 (socket says 172.21.15.49:48794) 2024-04-15T15:18:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.989+0000 7f7aca600640 1 --2- 172.21.15.49:0/2961557375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 0x7f7ac40d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.989+0000 7f7ac3e00640 1 -- 172.21.15.49:0/2961557375 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92971194 0 0) 0x7f7ac4160560 con 0x7f7ac40738f0 2024-04-15T15:18:37.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.989+0000 7f7ac3e00640 1 -- 172.21.15.49:0/2961557375 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7aac003520 con 0x7f7ac40738f0 2024-04-15T15:18:37.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.989+0000 7f7ac9c00640 1 -- 172.21.15.49:0/2961557375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 msgr2=0x7f7ac40d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:37.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.989+0000 7f7ac9c00640 1 --2- 172.21.15.49:0/2961557375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 0x7f7ac40d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.989+0000 7f7ac9c00640 1 -- 172.21.15.49:0/2961557375 >> v1:172.21.15.49:6789/0 conn(0x7f7ac40738f0 legacy=0x7f7ac40b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:37.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.989+0000 7f7ac9c00640 1 -- 172.21.15.49:0/2961557375 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac40cf8e0 con 0x7f7ac4158f20 2024-04-15T15:18:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.990+0000 7f7ac9c00640 1 --2- 172.21.15.49:0/2961557375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 0x7f7ac4160020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009870 tx=0x7f7ab802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f930434fba70a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.991+0000 7f7ac3e00640 1 -- 172.21.15.49:0/2961557375 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8034070 con 0x7f7ac4158f20 2024-04-15T15:18:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.991+0000 7f7ac3e00640 1 -- 172.21.15.49:0/2961557375 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab8030840 con 0x7f7ac4158f20 2024-04-15T15:18:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.991+0000 7f7ac3e00640 1 -- 172.21.15.49:0/2961557375 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8030c10 con 0x7f7ac4158f20 2024-04-15T15:18:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.991+0000 7f7acb000640 1 -- 172.21.15.49:0/2961557375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 msgr2=0x7f7ac4160020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.991+0000 7f7acb000640 1 --2- 172.21.15.49:0/2961557375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 0x7f7ac4160020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009870 tx=0x7f7ab802b980 comp rx=0 tx=0).stop 2024-04-15T15:18:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.991+0000 7f7acb000640 1 -- 172.21.15.49:0/2961557375 shutdown_connections 2024-04-15T15:18:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.991+0000 7f7acb000640 1 --2- 172.21.15.49:0/2961557375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 0x7f7ac40d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.991+0000 7f7acb000640 1 --2- 172.21.15.49:0/2961557375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 0x7f7ac4160020 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.991+0000 7f7acb000640 1 -- 172.21.15.49:0/2961557375 >> 172.21.15.49:0/2961557375 conn(0x7f7ac4067720 msgr2=0x7f7ac4067b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.991+0000 7f7acb000640 1 -- 172.21.15.49:0/2961557375 shutdown_connections 2024-04-15T15:18:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.992+0000 7f7acb000640 1 -- 172.21.15.49:0/2961557375 wait complete. 2024-04-15T15:18:37.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.993+0000 7f7acb000640 1 Processor -- start 2024-04-15T15:18:37.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.993+0000 7f7acb000640 1 -- start start 2024-04-15T15:18:37.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.994+0000 7f7acb000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac40738f0 0x7f7ac4145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.994+0000 7f7acb000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 0x7f7ac41f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.994+0000 7f7acb000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 0x7f7ac4148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.994+0000 7f7acb000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac40d0180 con 0x7f7ac4158f20 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.994+0000 7f7acb000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac40cfea0 con 0x7f7ac40738f0 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.994+0000 7f7acb000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac40cf2b0 con 0x7f7ac40b39a0 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7aca600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 0x7f7ac4148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7ac9200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 0x7f7ac41f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7aca600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 0x7f7ac4148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44966/0 (socket says 172.21.15.49:44966) 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7aca600640 1 -- 172.21.15.49:0/2629149847 learned_addr learned my addr 172.21.15.49:0/2629149847 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7ac9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac40738f0 0x7f7ac4145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7ac9200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 0x7f7ac41f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:51180/0 (socket says 172.21.15.49:51180) 2024-04-15T15:18:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7aca600640 1 -- 172.21.15.49:0/2629149847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 msgr2=0x7f7ac41f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7aca600640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 0x7f7ac41f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7aca600640 1 -- 172.21.15.49:0/2629149847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac40738f0 msgr2=0x7f7ac4145600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7aca600640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac40738f0 0x7f7ac4145600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.995+0000 7f7aca600640 1 -- 172.21.15.49:0/2629149847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac40cbe30 con 0x7f7ac4158f20 2024-04-15T15:18:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.996+0000 7f7aca600640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 0x7f7ac4148250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00efc0 tx=0x7f7abc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.996+0000 7f7ac1600640 1 -- 172.21.15.49:0/2629149847 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc018020 con 0x7f7ac4158f20 2024-04-15T15:18:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.996+0000 7f7ac1600640 1 -- 172.21.15.49:0/2629149847 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7abc014480 con 0x7f7ac4158f20 2024-04-15T15:18:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.996+0000 7f7ac1600640 1 -- 172.21.15.49:0/2629149847 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc014870 con 0x7f7ac4158f20 2024-04-15T15:18:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.996+0000 7f7acb000640 1 -- 172.21.15.49:0/2629149847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac40cc110 con 0x7f7ac4158f20 2024-04-15T15:18:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.996+0000 7f7acb000640 1 -- 172.21.15.49:0/2629149847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac40cc6d0 con 0x7f7ac4158f20 2024-04-15T15:18:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.997+0000 7f7acb000640 1 -- 172.21.15.49:0/2629149847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac40755c0 con 0x7f7ac4158f20 2024-04-15T15:18:38.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.998+0000 7f7ac1600640 1 -- 172.21.15.49:0/2629149847 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f7abc005d10 con 0x7f7ac4158f20 2024-04-15T15:18:38.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.999+0000 7f7ac1600640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7aa406c860 0x7f7aa406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:38.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.999+0000 7f7ac1600640 1 -- 172.21.15.49:0/2629149847 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4471+0+0 (secure 0 0 0) 0x7f7abc08c360 con 0x7f7ac4158f20 2024-04-15T15:18:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:37.999+0000 7f7ac9c00640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7aa406c860 0x7f7aa406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.000+0000 7f7ac9c00640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7aa406c860 0x7f7aa406ed20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8036000 tx=0x7f7ab803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:38.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.004+0000 7f7ac1600640 1 -- 172.21.15.49:0/2629149847 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7abc022030 con 0x7f7ac4158f20 2024-04-15T15:18:38.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.207+0000 7f7acb000640 1 -- 172.21.15.49:0/2629149847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ac41570b0 con 0x7f7ac4158f20 2024-04-15T15:18:38.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.208+0000 7f7ac1600640 1 -- 172.21.15.49:0/2629149847 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10909 (secure 0 0 0) 0x7f7abc010040 con 0x7f7ac4158f20 2024-04-15T15:18:38.211 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:38.211 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":14,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:37.367735+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 -- 172.21.15.49:0/2629149847 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7aa406c860 msgr2=0x7f7aa406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7aa406c860 0x7f7aa406ed20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8036000 tx=0x7f7ab803b040 comp rx=0 tx=0).stop 2024-04-15T15:18:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 -- 172.21.15.49:0/2629149847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 msgr2=0x7f7ac4148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 0x7f7ac4148250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00efc0 tx=0x7f7abc00c490 comp rx=0 tx=0).stop 2024-04-15T15:18:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 -- 172.21.15.49:0/2629149847 shutdown_connections 2024-04-15T15:18:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7aa406c860 0x7f7aa406ed20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ac40b39a0 0x7f7ac41f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ac4158f20 0x7f7ac4148250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 --2- 172.21.15.49:0/2629149847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac40738f0 0x7f7ac4145600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 -- 172.21.15.49:0/2629149847 >> 172.21.15.49:0/2629149847 conn(0x7f7ac4067720 msgr2=0x7f7ac41497b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 -- 172.21.15.49:0/2629149847 shutdown_connections 2024-04-15T15:18:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.211+0000 7f7acb000640 1 -- 172.21.15.49:0/2629149847 wait complete. 2024-04-15T15:18:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-15T15:18:38.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:18:38.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.454+0000 7ff090c00640 1 Processor -- start 2024-04-15T15:18:38.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.455+0000 7ff090c00640 1 -- start start 2024-04-15T15:18:38.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.456+0000 7ff090c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff08c0cee00 0x7ff08c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:38.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.456+0000 7ff090c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff08c071f50 0x7ff08c0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:38.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.456+0000 7ff090c00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff08c15dbb0 con 0x7ff08c15cbe0 2024-04-15T15:18:38.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.456+0000 7ff090c00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff08c06f4f0 con 0x7ff08c0cee00 2024-04-15T15:18:38.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.456+0000 7ff090c00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7ff08c0728f0 con 0x7ff08c071f50 2024-04-15T15:18:38.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.457+0000 7ff08be00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff08c15cbe0 0x7ff08c15d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48810/0 (socket says 172.21.15.49:48810) 2024-04-15T15:18:38.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.457+0000 7ff08be00640 1 -- 172.21.15.49:0/923855218 learned_addr learned my addr 172.21.15.49:0/923855218 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:38.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.457+0000 7ff08b400640 1 --2- 172.21.15.49:0/923855218 >> v2:172.21.15.49:3300/0 conn(0x7ff08c0cee00 0x7ff08c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:38.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.457+0000 7ff082a00640 1 --2- 172.21.15.49:0/923855218 >> v2:172.21.15.107:3300/0 conn(0x7ff08c071f50 0x7ff08c0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:38.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.457+0000 7ff08aa00640 1 -- 172.21.15.49:0/923855218 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 792311003 0 0) 0x7ff08c15dbb0 con 0x7ff08c15cbe0 2024-04-15T15:18:38.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.457+0000 7ff08aa00640 1 -- 172.21.15.49:0/923855218 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff070003520 con 0x7ff08c15cbe0 2024-04-15T15:18:38.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.457+0000 7ff08b400640 1 -- 172.21.15.49:0/923855218 >> v2:172.21.15.107:3300/0 conn(0x7ff08c071f50 msgr2=0x7ff08c0723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:38.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.457+0000 7ff08b400640 1 --2- 172.21.15.49:0/923855218 >> v2:172.21.15.107:3300/0 conn(0x7ff08c071f50 0x7ff08c0723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.457+0000 7ff08b400640 1 -- 172.21.15.49:0/923855218 >> v1:172.21.15.49:6789/0 conn(0x7ff08c15cbe0 legacy=0x7ff08c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:38.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.457+0000 7ff08b400640 1 -- 172.21.15.49:0/923855218 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c15d8d0 con 0x7ff08c0cee00 2024-04-15T15:18:38.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.459+0000 7ff08b400640 1 --2- 172.21.15.49:0/923855218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 0x7ff08c06e8c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0b5aa0 tx=0x7ff07c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b627441ec187b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:38.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.459+0000 7ff08aa00640 1 -- 172.21.15.49:0/923855218 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c035020 con 0x7ff08c0cee00 2024-04-15T15:18:38.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.459+0000 7ff08aa00640 1 -- 172.21.15.49:0/923855218 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07c031830 con 0x7ff08c0cee00 2024-04-15T15:18:38.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.459+0000 7ff08aa00640 1 -- 172.21.15.49:0/923855218 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c031c00 con 0x7ff08c0cee00 2024-04-15T15:18:38.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.459+0000 7ff090c00640 1 -- 172.21.15.49:0/923855218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 msgr2=0x7ff08c06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:38.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.459+0000 7ff090c00640 1 --2- 172.21.15.49:0/923855218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 0x7ff08c06e8c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0b5aa0 tx=0x7ff07c02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.460+0000 7ff090c00640 1 -- 172.21.15.49:0/923855218 shutdown_connections 2024-04-15T15:18:38.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.460+0000 7ff090c00640 1 --2- 172.21.15.49:0/923855218 >> v2:172.21.15.107:3300/0 conn(0x7ff08c071f50 0x7ff08c0723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.460+0000 7ff090c00640 1 --2- 172.21.15.49:0/923855218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 0x7ff08c06e8c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.460+0000 7ff090c00640 1 -- 172.21.15.49:0/923855218 >> 172.21.15.49:0/923855218 conn(0x7ff08c06b930 msgr2=0x7ff08c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:38.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.460+0000 7ff090c00640 1 -- 172.21.15.49:0/923855218 shutdown_connections 2024-04-15T15:18:38.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.460+0000 7ff090c00640 1 -- 172.21.15.49:0/923855218 wait complete. 2024-04-15T15:18:38.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.461+0000 7ff090c00640 1 Processor -- start 2024-04-15T15:18:38.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.462+0000 7ff090c00640 1 -- start start 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff090c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff08c071f50 0x7ff08c153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff090c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 0x7ff08c154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff090c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff08c15cbe0 0x7ff08c14cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff090c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c06e3a0 con 0x7ff08c0cee00 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff090c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c06e0c0 con 0x7ff08c15cbe0 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff090c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff08c15dda0 con 0x7ff08c071f50 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff082a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 0x7ff08c154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff082a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 0x7ff08c154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44976/0 (socket says 172.21.15.49:44976) 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff082a00640 1 -- 172.21.15.49:0/316199832 learned_addr learned my addr 172.21.15.49:0/316199832 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff08b400640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff08c071f50 0x7ff08c153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:38.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff08be00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff08c15cbe0 0x7ff08c14cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff082a00640 1 -- 172.21.15.49:0/316199832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff08c071f50 msgr2=0x7ff08c153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff082a00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff08c071f50 0x7ff08c153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff082a00640 1 -- 172.21.15.49:0/316199832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff08c15cbe0 msgr2=0x7ff08c14cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff082a00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff08c15cbe0 0x7ff08c14cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.463+0000 7ff082a00640 1 -- 172.21.15.49:0/316199832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c1523b0 con 0x7ff08c0cee00 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.464+0000 7ff082a00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 0x7ff08c154310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff07400e9b0 tx=0x7ff07400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.464+0000 7ff083e00640 1 -- 172.21.15.49:0/316199832 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074017070 con 0x7ff08c0cee00 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.464+0000 7ff083e00640 1 -- 172.21.15.49:0/316199832 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07400c9a0 con 0x7ff08c0cee00 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.464+0000 7ff083e00640 1 -- 172.21.15.49:0/316199832 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07400cd90 con 0x7ff08c0cee00 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.464+0000 7ff090c00640 1 -- 172.21.15.49:0/316199832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c152690 con 0x7ff08c0cee00 2024-04-15T15:18:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.464+0000 7ff090c00640 1 -- 172.21.15.49:0/316199832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c152c50 con 0x7ff08c0cee00 2024-04-15T15:18:38.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.466+0000 7ff090c00640 1 -- 172.21.15.49:0/316199832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c0755c0 con 0x7ff08c0cee00 2024-04-15T15:18:38.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.466+0000 7ff083e00640 1 -- 172.21.15.49:0/316199832 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ff07401a070 con 0x7ff08c0cee00 2024-04-15T15:18:38.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.467+0000 7ff083e00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff06806c860 0x7ff06806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:38.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.467+0000 7ff083e00640 1 -- 172.21.15.49:0/316199832 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4471+0+0 (secure 0 0 0) 0x7ff07408b940 con 0x7ff08c0cee00 2024-04-15T15:18:38.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.468+0000 7ff08b400640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff06806c860 0x7ff06806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:38.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.468+0000 7ff08b400640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff06806c860 0x7ff06806ed20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff07c036000 tx=0x7ff07c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.472+0000 7ff083e00640 1 -- 172.21.15.49:0/316199832 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff07405a350 con 0x7ff08c0cee00 2024-04-15T15:18:38.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.719+0000 7ff090c00640 1 -- 172.21.15.49:0/316199832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff08c06fb00 con 0x7ff08c0cee00 2024-04-15T15:18:38.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.720+0000 7ff083e00640 1 -- 172.21.15.49:0/316199832 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7ff074059c50 con 0x7ff08c0cee00 2024-04-15T15:18:38.723 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:38.723 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3198352923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3198352923},{"type":"v1","addr":"172.21.15.49:6836","nonce":3198352923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-15T15:18:38.723 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.722+0000 7ff090c00640 1 -- 172.21.15.49:0/316199832 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff06806c860 msgr2=0x7ff06806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.722+0000 7ff090c00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff06806c860 0x7ff06806ed20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff07c036000 tx=0x7ff07c03c040 comp rx=0 tx=0).stop 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.723+0000 7ff090c00640 1 -- 172.21.15.49:0/316199832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 msgr2=0x7ff08c154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.723+0000 7ff090c00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 0x7ff08c154310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff07400e9b0 tx=0x7ff07400ee70 comp rx=0 tx=0).stop 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.723+0000 7ff090c00640 1 -- 172.21.15.49:0/316199832 shutdown_connections 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.723+0000 7ff090c00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff06806c860 0x7ff06806ed20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.723+0000 7ff090c00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff08c15cbe0 0x7ff08c14cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.723+0000 7ff090c00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff08c0cee00 0x7ff08c154310 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.723+0000 7ff090c00640 1 --2- 172.21.15.49:0/316199832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff08c071f50 0x7ff08c153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.723+0000 7ff090c00640 1 -- 172.21.15.49:0/316199832 >> 172.21.15.49:0/316199832 conn(0x7ff08c06b930 msgr2=0x7ff08c159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.723+0000 7ff090c00640 1 -- 172.21.15.49:0/316199832 shutdown_connections 2024-04-15T15:18:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:38.723+0000 7ff090c00640 1 -- 172.21.15.49:0/316199832 wait complete. 2024-04-15T15:18:40.744 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:18:40.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.979+0000 7f74f3200640 1 Processor -- start 2024-04-15T15:18:40.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.979+0000 7f74f3200640 1 -- start start 2024-04-15T15:18:40.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.981+0000 7f74f3200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74ec072660 0x7f74ec06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:40.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.981+0000 7f74f3200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f74ec15cbe0 0x7f74ec15d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:40.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.981+0000 7f74f3200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74ec15dbb0 con 0x7f74ec0cee00 2024-04-15T15:18:40.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.981+0000 7f74f3200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f74ec06f4f0 con 0x7f74ec072660 2024-04-15T15:18:40.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.981+0000 7f74f3200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f74ec15d8d0 con 0x7f74ec15cbe0 2024-04-15T15:18:40.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.982+0000 7f74f1e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f74ec0cee00 0x7f74ec071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48814/0 (socket says 172.21.15.49:48814) 2024-04-15T15:18:40.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.982+0000 7f74f1e00640 1 -- 172.21.15.49:0/3788591767 learned_addr learned my addr 172.21.15.49:0/3788591767 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:40.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.982+0000 7f74f1400640 1 --2- 172.21.15.49:0/3788591767 >> v2:172.21.15.49:3300/0 conn(0x7f74ec072660 0x7f74ec06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:40.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.982+0000 7f74f2800640 1 --2- 172.21.15.49:0/3788591767 >> v2:172.21.15.107:3300/0 conn(0x7f74ec15cbe0 0x7f74ec15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:40.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.982+0000 7f74f0a00640 1 -- 172.21.15.49:0/3788591767 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1165027733 0 0) 0x7f74ec15dbb0 con 0x7f74ec0cee00 2024-04-15T15:18:40.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.982+0000 7f74f0a00640 1 -- 172.21.15.49:0/3788591767 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74d4003520 con 0x7f74ec0cee00 2024-04-15T15:18:40.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.982+0000 7f74f1400640 1 -- 172.21.15.49:0/3788591767 >> v2:172.21.15.107:3300/0 conn(0x7f74ec15cbe0 msgr2=0x7f74ec15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:40.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.982+0000 7f74f1400640 1 --2- 172.21.15.49:0/3788591767 >> v2:172.21.15.107:3300/0 conn(0x7f74ec15cbe0 0x7f74ec15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:40.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.982+0000 7f74f1400640 1 -- 172.21.15.49:0/3788591767 >> v1:172.21.15.49:6789/0 conn(0x7f74ec0cee00 legacy=0x7f74ec071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:40.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.982+0000 7f74f1400640 1 -- 172.21.15.49:0/3788591767 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ec06dde0 con 0x7f74ec072660 2024-04-15T15:18:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.984+0000 7f74f1400640 1 --2- 172.21.15.49:0/3788591767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 0x7f74ec06e8c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0b02a0 tx=0x7f74dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9cc35071460b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.984+0000 7f74f0a00640 1 -- 172.21.15.49:0/3788591767 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc035020 con 0x7f74ec072660 2024-04-15T15:18:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.984+0000 7f74f0a00640 1 -- 172.21.15.49:0/3788591767 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74dc031830 con 0x7f74ec072660 2024-04-15T15:18:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.984+0000 7f74f0a00640 1 -- 172.21.15.49:0/3788591767 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc031c00 con 0x7f74ec072660 2024-04-15T15:18:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.985+0000 7f74f3200640 1 -- 172.21.15.49:0/3788591767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 msgr2=0x7f74ec06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.985+0000 7f74f3200640 1 --2- 172.21.15.49:0/3788591767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 0x7f74ec06e8c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0b02a0 tx=0x7f74dc02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:18:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.985+0000 7f74f3200640 1 -- 172.21.15.49:0/3788591767 shutdown_connections 2024-04-15T15:18:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.985+0000 7f74f3200640 1 --2- 172.21.15.49:0/3788591767 >> v2:172.21.15.107:3300/0 conn(0x7f74ec15cbe0 0x7f74ec15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.985+0000 7f74f3200640 1 --2- 172.21.15.49:0/3788591767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 0x7f74ec06e8c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.985+0000 7f74f3200640 1 -- 172.21.15.49:0/3788591767 >> 172.21.15.49:0/3788591767 conn(0x7f74ec06b930 msgr2=0x7f74ec06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.985+0000 7f74f3200640 1 -- 172.21.15.49:0/3788591767 shutdown_connections 2024-04-15T15:18:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.985+0000 7f74f3200640 1 -- 172.21.15.49:0/3788591767 wait complete. 2024-04-15T15:18:40.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.987+0000 7f74f3200640 1 Processor -- start 2024-04-15T15:18:40.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.987+0000 7f74f3200640 1 -- start start 2024-04-15T15:18:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.988+0000 7f74f3200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 0x7f74ec155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.988+0000 7f74f3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74ec0cee00 0x7f74ec15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.988+0000 7f74f3200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74ec15cbe0 0x7f74ec143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.988+0000 7f74f3200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec15d5b0 con 0x7f74ec072660 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.988+0000 7f74f3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec06e3a0 con 0x7f74ec0cee00 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.988+0000 7f74f3200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f74ec15dda0 con 0x7f74ec15cbe0 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.988+0000 7f74f1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 0x7f74ec155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.988+0000 7f74f1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 0x7f74ec155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44994/0 (socket says 172.21.15.49:44994) 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.988+0000 7f74f1e00640 1 -- 172.21.15.49:0/3178808383 learned_addr learned my addr 172.21.15.49:0/3178808383 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f1400640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74ec0cee00 0x7f74ec15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f2800640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74ec15cbe0 0x7f74ec143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f1e00640 1 -- 172.21.15.49:0/3178808383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74ec15cbe0 msgr2=0x7f74ec143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f1e00640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74ec15cbe0 0x7f74ec143e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f1e00640 1 -- 172.21.15.49:0/3178808383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74ec0cee00 msgr2=0x7f74ec15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f1e00640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74ec0cee00 0x7f74ec15a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f1e00640 1 -- 172.21.15.49:0/3178808383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ec15b050 con 0x7f74ec072660 2024-04-15T15:18:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f2800640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74ec15cbe0 0x7f74ec143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f1400640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74ec0cee00 0x7f74ec15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f1e00640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 0x7f74ec155080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f74e000c430 tx=0x7f74e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74e6000640 1 -- 172.21.15.49:0/3178808383 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0017070 con 0x7f74ec072660 2024-04-15T15:18:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74e6000640 1 -- 172.21.15.49:0/3178808383 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e0005ce0 con 0x7f74ec072660 2024-04-15T15:18:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74e6000640 1 -- 172.21.15.49:0/3178808383 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e00072f0 con 0x7f74ec072660 2024-04-15T15:18:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.989+0000 7f74f3200640 1 -- 172.21.15.49:0/3178808383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ec15b330 con 0x7f74ec072660 2024-04-15T15:18:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.990+0000 7f74f3200640 1 -- 172.21.15.49:0/3178808383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ec15b8f0 con 0x7f74ec072660 2024-04-15T15:18:40.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.991+0000 7f74f3200640 1 -- 172.21.15.49:0/3178808383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ec0755c0 con 0x7f74ec072660 2024-04-15T15:18:40.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.992+0000 7f74e6000640 1 -- 172.21.15.49:0/3178808383 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f74e001a070 con 0x7f74ec072660 2024-04-15T15:18:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.993+0000 7f74e6000640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f74d006c860 0x7f74d006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.993+0000 7f74e6000640 1 -- 172.21.15.49:0/3178808383 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f74e008ac10 con 0x7f74ec072660 2024-04-15T15:18:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.993+0000 7f74f1400640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f74d006c860 0x7f74d006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.994+0000 7f74f1400640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f74d006c860 0x7f74d006ed20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f74dc036000 tx=0x7f74dc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:41.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:40.999+0000 7f74e6000640 1 -- 172.21.15.49:0/3178808383 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74e0059460 con 0x7f74ec072660 2024-04-15T15:18:41.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.251+0000 7f74f3200640 1 -- 172.21.15.49:0/3178808383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74ec0ae6d0 con 0x7f74ec072660 2024-04-15T15:18:41.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.253+0000 7f74e6000640 1 -- 172.21.15.49:0/3178808383 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f74e0058d60 con 0x7f74ec072660 2024-04-15T15:18:41.255 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:41.256 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/203558139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":203558139},{"type":"v1","addr":"172.21.15.107:6833","nonce":203558139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3198352923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3198352923},{"type":"v1","addr":"172.21.15.49:6836","nonce":3198352923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1425337425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1425337425},{"type":"v1","addr":"172.21.15.49:6837","nonce":1425337425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2374398161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2374398161},{"type":"v1","addr":"172.21.15.107:6835","nonce":2374398161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-15T15:18:41.256 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2024-04-15T15:18:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.255+0000 7f74f3200640 1 -- 172.21.15.49:0/3178808383 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f74d006c860 msgr2=0x7f74d006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.255+0000 7f74f3200640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f74d006c860 0x7f74d006ed20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f74dc036000 tx=0x7f74dc008030 comp rx=0 tx=0).stop 2024-04-15T15:18:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.255+0000 7f74f3200640 1 -- 172.21.15.49:0/3178808383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 msgr2=0x7f74ec155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.255+0000 7f74f3200640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 0x7f74ec155080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f74e000c430 tx=0x7f74e000c8f0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.255+0000 7f74f3200640 1 -- 172.21.15.49:0/3178808383 shutdown_connections 2024-04-15T15:18:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.255+0000 7f74f3200640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f74d006c860 0x7f74d006ed20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.255+0000 7f74f3200640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74ec15cbe0 0x7f74ec143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.255+0000 7f74f3200640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74ec0cee00 0x7f74ec15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.255+0000 7f74f3200640 1 --2- 172.21.15.49:0/3178808383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ec072660 0x7f74ec155080 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.255+0000 7f74f3200640 1 -- 172.21.15.49:0/3178808383 >> 172.21.15.49:0/3178808383 conn(0x7f74ec06b930 msgr2=0x7f74ec15e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:41.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.256+0000 7f74f3200640 1 -- 172.21.15.49:0/3178808383 shutdown_connections 2024-04-15T15:18:41.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.256+0000 7f74f3200640 1 -- 172.21.15.49:0/3178808383 wait complete. 2024-04-15T15:18:41.279 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:41.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.488+0000 7f722b000640 1 Processor -- start 2024-04-15T15:18:41.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.488+0000 7f722b000640 1 -- start start 2024-04-15T15:18:41.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.490+0000 7f722b000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f722406cb50 0x7f7224073180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.490+0000 7f722b000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72240736c0 0x7f72240aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.490+0000 7f722b000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7224162760 con 0x7f7224159240 2024-04-15T15:18:41.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.490+0000 7f722b000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f72240cd8f0 con 0x7f72240736c0 2024-04-15T15:18:41.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.490+0000 7f722b000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f72241629d0 con 0x7f722406cb50 2024-04-15T15:18:41.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.490+0000 7f7223e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7224159240 0x7f722415fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48820/0 (socket says 172.21.15.49:48820) 2024-04-15T15:18:41.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.490+0000 7f7223e00640 1 -- 172.21.15.49:0/2975145666 learned_addr learned my addr 172.21.15.49:0/2975145666 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:41.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.490+0000 7f7229000640 1 --2- 172.21.15.49:0/2975145666 >> v2:172.21.15.49:3300/0 conn(0x7f72240736c0 0x7f72240aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.490+0000 7f7223400640 1 --2- 172.21.15.49:0/2975145666 >> v2:172.21.15.107:3300/0 conn(0x7f722406cb50 0x7f7224073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.491+0000 7f7222a00640 1 -- 172.21.15.49:0/2975145666 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4223925563 0 0) 0x7f7224162760 con 0x7f7224159240 2024-04-15T15:18:41.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.491+0000 7f7222a00640 1 -- 172.21.15.49:0/2975145666 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7208003520 con 0x7f7224159240 2024-04-15T15:18:41.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.491+0000 7f7229000640 1 -- 172.21.15.49:0/2975145666 >> v2:172.21.15.107:3300/0 conn(0x7f722406cb50 msgr2=0x7f7224073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:41.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.491+0000 7f7229000640 1 --2- 172.21.15.49:0/2975145666 >> v2:172.21.15.107:3300/0 conn(0x7f722406cb50 0x7f7224073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.491+0000 7f7229000640 1 -- 172.21.15.49:0/2975145666 >> v1:172.21.15.49:6789/0 conn(0x7f7224159240 legacy=0x7f722415fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.491+0000 7f7229000640 1 -- 172.21.15.49:0/2975145666 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7224162cb0 con 0x7f72240736c0 2024-04-15T15:18:41.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.492+0000 7f7229000640 1 --2- 172.21.15.49:0/2975145666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72240736c0 0x7f72240aed10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f722406d440 tx=0x7f721c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b41e6dafa52beb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.492+0000 7f7222a00640 1 -- 172.21.15.49:0/2975145666 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c039070 con 0x7f72240736c0 2024-04-15T15:18:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.492+0000 7f7222a00640 1 -- 172.21.15.49:0/2975145666 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f721c02cc80 con 0x7f72240736c0 2024-04-15T15:18:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.493+0000 7f7222a00640 1 -- 172.21.15.49:0/2975145666 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c034660 con 0x7f72240736c0 2024-04-15T15:18:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.493+0000 7f722b000640 1 -- 172.21.15.49:0/2975145666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72240736c0 msgr2=0x7f72240aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.493+0000 7f722b000640 1 --2- 172.21.15.49:0/2975145666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72240736c0 0x7f72240aed10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f722406d440 tx=0x7f721c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.493+0000 7f722b000640 1 -- 172.21.15.49:0/2975145666 shutdown_connections 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.493+0000 7f722b000640 1 --2- 172.21.15.49:0/2975145666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72240736c0 0x7f72240aed10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.493+0000 7f722b000640 1 --2- 172.21.15.49:0/2975145666 >> v2:172.21.15.107:3300/0 conn(0x7f722406cb50 0x7f7224073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.493+0000 7f722b000640 1 -- 172.21.15.49:0/2975145666 >> 172.21.15.49:0/2975145666 conn(0x7f722406f860 msgr2=0x7f7224072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.493+0000 7f722b000640 1 -- 172.21.15.49:0/2975145666 shutdown_connections 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.493+0000 7f722b000640 1 -- 172.21.15.49:0/2975145666 wait complete. 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.495+0000 7f722b000640 1 Processor -- start 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.495+0000 7f722b000640 1 -- start start 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.496+0000 7f722b000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f722406cb50 0x7f72240cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.496+0000 7f722b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72240736c0 0x7f7224145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.496+0000 7f722b000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7224159240 0x7f72241f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.496+0000 7f722b000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f72240af3d0 con 0x7f722406cb50 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.496+0000 7f722b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72240af250 con 0x7f72240736c0 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.496+0000 7f722b000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f72240af550 con 0x7f7224159240 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7223e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f722406cb50 0x7f72240cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7223e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f722406cb50 0x7f72240cdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45028/0 (socket says 172.21.15.49:45028) 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7223e00640 1 -- 172.21.15.49:0/479808341 learned_addr learned my addr 172.21.15.49:0/479808341 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7229000640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7224159240 0x7f72241f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7223400640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72240736c0 0x7f7224145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7223e00640 1 -- 172.21.15.49:0/479808341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7224159240 msgr2=0x7f72241f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7223e00640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7224159240 0x7f72241f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7223e00640 1 -- 172.21.15.49:0/479808341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72240736c0 msgr2=0x7f7224145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7223e00640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72240736c0 0x7f7224145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7223e00640 1 -- 172.21.15.49:0/479808341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72240cfca0 con 0x7f722406cb50 2024-04-15T15:18:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.497+0000 7f7223e00640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f722406cb50 0x7f72240cdb00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f721400c390 tx=0x7f721400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:41.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.498+0000 7f721be00640 1 -- 172.21.15.49:0/479808341 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214017070 con 0x7f722406cb50 2024-04-15T15:18:41.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.498+0000 7f721be00640 1 -- 172.21.15.49:0/479808341 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7214005ce0 con 0x7f722406cb50 2024-04-15T15:18:41.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.498+0000 7f722b000640 1 -- 172.21.15.49:0/479808341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72240ce0e0 con 0x7f722406cb50 2024-04-15T15:18:41.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.498+0000 7f721be00640 1 -- 172.21.15.49:0/479808341 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72140072f0 con 0x7f722406cb50 2024-04-15T15:18:41.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.498+0000 7f722b000640 1 -- 172.21.15.49:0/479808341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72240ce6a0 con 0x7f722406cb50 2024-04-15T15:18:41.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.499+0000 7f722b000640 1 -- 172.21.15.49:0/479808341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7224068c80 con 0x7f722406cb50 2024-04-15T15:18:41.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.500+0000 7f721be00640 1 -- 172.21.15.49:0/479808341 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f721401a070 con 0x7f722406cb50 2024-04-15T15:18:41.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.501+0000 7f721be00640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f720406c890 0x7f720406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.501+0000 7f721be00640 1 -- 172.21.15.49:0/479808341 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f721408aae0 con 0x7f722406cb50 2024-04-15T15:18:41.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.501+0000 7f7223400640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f720406c890 0x7f720406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.502+0000 7f7223400640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f720406c890 0x7f720406ed50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f722406c430 tx=0x7f7210008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.506+0000 7f721be00640 1 -- 172.21.15.49:0/479808341 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72240cfca0 con 0x7f722406cb50 2024-04-15T15:18:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.719+0000 7f722b000640 1 -- 172.21.15.49:0/479808341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72240680c0 con 0x7f722406cb50 2024-04-15T15:18:41.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.721+0000 7f721be00640 1 -- 172.21.15.49:0/479808341 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12884 (secure 0 0 0) 0x7f7214059340 con 0x7f722406cb50 2024-04-15T15:18:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:41.724 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":18,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:41.669310+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:41.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.723+0000 7f722b000640 1 -- 172.21.15.49:0/479808341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f720406c890 msgr2=0x7f720406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.723+0000 7f722b000640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f720406c890 0x7f720406ed50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f722406c430 tx=0x7f7210008040 comp rx=0 tx=0).stop 2024-04-15T15:18:41.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.723+0000 7f722b000640 1 -- 172.21.15.49:0/479808341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f722406cb50 msgr2=0x7f72240cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.723+0000 7f722b000640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f722406cb50 0x7f72240cdb00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f721400c390 tx=0x7f721400c850 comp rx=0 tx=0).stop 2024-04-15T15:18:41.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.723+0000 7f722b000640 1 -- 172.21.15.49:0/479808341 shutdown_connections 2024-04-15T15:18:41.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.723+0000 7f722b000640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f720406c890 0x7f720406ed50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.723+0000 7f722b000640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7224159240 0x7f72241f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.723+0000 7f722b000640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72240736c0 0x7f7224145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.723+0000 7f722b000640 1 --2- 172.21.15.49:0/479808341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f722406cb50 0x7f72240cdb00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.723+0000 7f722b000640 1 -- 172.21.15.49:0/479808341 >> 172.21.15.49:0/479808341 conn(0x7f722406f860 msgr2=0x7f72240b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:41.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.724+0000 7f722b000640 1 -- 172.21.15.49:0/479808341 shutdown_connections 2024-04-15T15:18:41.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.724+0000 7f722b000640 1 -- 172.21.15.49:0/479808341 wait complete. 2024-04-15T15:18:41.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:18:41.745 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:18:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.946+0000 7f616de00640 1 Processor -- start 2024-04-15T15:18:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.947+0000 7f616de00640 1 -- start start 2024-04-15T15:18:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.949+0000 7f616de00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f616815edd0 0x7f61680ae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.949+0000 7f616de00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f61680cb4b0 0x7f61680ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.949+0000 7f616de00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61680b1be0 con 0x7f61680b07a0 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.949+0000 7f616de00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f61680b1ec0 con 0x7f61680cb4b0 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.949+0000 7f616de00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f61680cf850 con 0x7f616815edd0 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.949+0000 7f6167400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f61680b07a0 0x7f61680d0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48826/0 (socket says 172.21.15.49:48826) 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6167400640 1 -- 172.21.15.49:0/130841713 learned_addr learned my addr 172.21.15.49:0/130841713 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6166a00640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f616815edd0 0x7f61680ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6166000640 1 --2- 172.21.15.49:0/130841713 >> v2:172.21.15.107:3300/0 conn(0x7f61680cb4b0 0x7f61680ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4142725538 0 0) 0x7f61680b1be0 con 0x7f61680b07a0 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f614c003520 con 0x7f61680b07a0 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2115882633 0 0) 0x7f614c003520 con 0x7f61680b07a0 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f616815edd0 msgr2=0x7f61680ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6165600640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f616815edd0 0x7f61680ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 >> v2:172.21.15.107:3300/0 conn(0x7f61680cb4b0 msgr2=0x7f61680ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6165600640 1 --2- 172.21.15.49:0/130841713 >> v2:172.21.15.107:3300/0 conn(0x7f61680cb4b0 0x7f61680ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61680cfb30 con 0x7f61680b07a0 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f61600019f0 con 0x7f61680b07a0 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6166000640 1 --2- 172.21.15.49:0/130841713 >> v2:172.21.15.107:3300/0 conn(0x7f61680cb4b0 0x7f61680ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.950+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 >> v1:172.21.15.49:6789/0 conn(0x7f61680b07a0 legacy=0x7f61680d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.951+0000 7f6165600640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f614c0039b0 0x7f614c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.951+0000 7f6165600640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f614c004310 0x7f614c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.951+0000 7f6165600640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f614c004f00 0x7f614c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.951+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61680b1ec0 con 0x7f614c004f00 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.951+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61680cf850 con 0x7f614c004310 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.951+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f614c00bf00 con 0x7f614c0039b0 2024-04-15T15:18:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.951+0000 7f6167400640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f614c004f00 0x7f614c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.951+0000 7f6166a00640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f614c0039b0 0x7f614c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.951+0000 7f6166000640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f614c004310 0x7f614c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.952+0000 7f6167400640 1 -- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f614c0039b0 msgr2=0x7f614c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.952+0000 7f6167400640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f614c0039b0 0x7f614c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.952+0000 7f6167400640 1 -- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f614c004310 msgr2=0x7f614c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.952+0000 7f6167400640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f614c004310 0x7f614c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.952+0000 7f6167400640 1 -- 172.21.15.49:0/130841713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c003520 con 0x7f614c004f00 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.954+0000 7f6167400640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f614c004f00 0x7f614c00b9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f616000a920 tx=0x7f614c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd53a9d70a44b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.954+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616000a4c0 con 0x7f614c004f00 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.954+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6160034650 con 0x7f614c004f00 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.954+0000 7f6165600640 1 -- 172.21.15.49:0/130841713 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160034a80 con 0x7f614c004f00 2024-04-15T15:18:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.955+0000 7f616de00640 1 -- 172.21.15.49:0/130841713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f614c004f00 msgr2=0x7f614c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.955+0000 7f616de00640 1 --2- 172.21.15.49:0/130841713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f614c004f00 0x7f614c00b9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f616000a920 tx=0x7f614c005830 comp rx=0 tx=0).stop 2024-04-15T15:18:41.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.955+0000 7f6167400640 1 -- 172.21.15.49:0/130841713 reap_dead start 2024-04-15T15:18:41.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.955+0000 7f616de00640 1 -- 172.21.15.49:0/130841713 shutdown_connections 2024-04-15T15:18:41.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.955+0000 7f616de00640 1 -- 172.21.15.49:0/130841713 >> 172.21.15.49:0/130841713 conn(0x7f61680b3290 msgr2=0x7f61680b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:41.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.955+0000 7f616de00640 1 -- 172.21.15.49:0/130841713 shutdown_connections 2024-04-15T15:18:41.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.955+0000 7f616de00640 1 -- 172.21.15.49:0/130841713 wait complete. 2024-04-15T15:18:41.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.957+0000 7f616de00640 1 Processor -- start 2024-04-15T15:18:41.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.957+0000 7f616de00640 1 -- start start 2024-04-15T15:18:41.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.958+0000 7f616de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6168150350 0x7f61680b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.958+0000 7f616de00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6168150730 0x7f616815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.958+0000 7f616de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61681f7790 0x7f616815de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.958+0000 7f616de00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61680cf5f0 con 0x7f6168150350 2024-04-15T15:18:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.958+0000 7f616de00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61680cf470 con 0x7f61681f7790 2024-04-15T15:18:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.958+0000 7f616de00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f61680d0760 con 0x7f6168150730 2024-04-15T15:18:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.958+0000 7f6166a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6168150350 0x7f61680b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.958+0000 7f6166a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6168150350 0x7f61680b62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45042/0 (socket says 172.21.15.49:45042) 2024-04-15T15:18:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.958+0000 7f6166a00640 1 -- 172.21.15.49:0/1104464403 learned_addr learned my addr 172.21.15.49:0/1104464403 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f6167400640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61681f7790 0x7f616815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f6166000640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6168150730 0x7f616815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f6166a00640 1 -- 172.21.15.49:0/1104464403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6168150730 msgr2=0x7f616815d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f6166a00640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6168150730 0x7f616815d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f6166a00640 1 -- 172.21.15.49:0/1104464403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61681f7790 msgr2=0x7f616815de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f6166a00640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61681f7790 0x7f616815de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f6166a00640 1 -- 172.21.15.49:0/1104464403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681f7ce0 con 0x7f6168150350 2024-04-15T15:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f6166a00640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6168150350 0x7f61680b62c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61580118e0 tx=0x7f6158011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f615f400640 1 -- 172.21.15.49:0/1104464403 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615800faf0 con 0x7f6168150350 2024-04-15T15:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f615f400640 1 -- 172.21.15.49:0/1104464403 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615800fc90 con 0x7f6168150350 2024-04-15T15:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f615f400640 1 -- 172.21.15.49:0/1104464403 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158014440 con 0x7f6168150350 2024-04-15T15:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.959+0000 7f616de00640 1 -- 172.21.15.49:0/1104464403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681f7fc0 con 0x7f6168150350 2024-04-15T15:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.960+0000 7f616de00640 1 -- 172.21.15.49:0/1104464403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61681f8580 con 0x7f6168150350 2024-04-15T15:18:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.961+0000 7f616de00640 1 -- 172.21.15.49:0/1104464403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6168068c80 con 0x7f6168150350 2024-04-15T15:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.962+0000 7f615f400640 1 -- 172.21.15.49:0/1104464403 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f6158012040 con 0x7f6168150350 2024-04-15T15:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.963+0000 7f615f400640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f614806c890 0x7f614806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.963+0000 7f615f400640 1 -- 172.21.15.49:0/1104464403 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f615808bdf0 con 0x7f6168150350 2024-04-15T15:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.963+0000 7f6166000640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f614806c890 0x7f614806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.964+0000 7f6166000640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f614806c890 0x7f614806ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61680b76b0 tx=0x7f6154008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:41.968+0000 7f615f400640 1 -- 172.21.15.49:0/1104464403 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6158019030 con 0x7f6168150350 2024-04-15T15:18:42.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.176+0000 7f616de00640 1 -- 172.21.15.49:0/1104464403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61680ae880 con 0x7f6168150350 2024-04-15T15:18:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.177+0000 7f615f400640 1 -- 172.21.15.49:0/1104464403 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7f615805a640 con 0x7f6168150350 2024-04-15T15:18:42.180 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:18:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 -- 172.21.15.49:0/1104464403 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f614806c890 msgr2=0x7f614806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f614806c890 0x7f614806ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61680b76b0 tx=0x7f6154008040 comp rx=0 tx=0).stop 2024-04-15T15:18:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 -- 172.21.15.49:0/1104464403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6168150350 msgr2=0x7f61680b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6168150350 0x7f61680b62c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61580118e0 tx=0x7f6158011da0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 -- 172.21.15.49:0/1104464403 shutdown_connections 2024-04-15T15:18:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f614806c890 0x7f614806ed50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6168150730 0x7f616815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61681f7790 0x7f616815de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 --2- 172.21.15.49:0/1104464403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6168150350 0x7f61680b62c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 -- 172.21.15.49:0/1104464403 >> 172.21.15.49:0/1104464403 conn(0x7f61680b3290 msgr2=0x7f61680cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:42.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 -- 172.21.15.49:0/1104464403 shutdown_connections 2024-04-15T15:18:42.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.180+0000 7f616de00640 1 -- 172.21.15.49:0/1104464403 wait complete. 2024-04-15T15:18:42.202 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-15T15:18:42.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.408+0000 7fbb95000640 1 Processor -- start 2024-04-15T15:18:42.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.409+0000 7fbb95000640 1 -- start start 2024-04-15T15:18:42.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.410+0000 7fbb95000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbb900cee00 0x7fbb9006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.410+0000 7fbb95000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9015dbb0 con 0x7fbb9015d2f0 2024-04-15T15:18:42.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.410+0000 7fbb95000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9015d8d0 con 0x7fbb90071f50 2024-04-15T15:18:42.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.410+0000 7fbb95000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fbb9006f4f0 con 0x7fbb900cee00 2024-04-15T15:18:42.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.410+0000 7fbb8fe00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fbb9015d2f0 0x7fbb9015dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48838/0 (socket says 172.21.15.49:48838) 2024-04-15T15:18:42.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.410+0000 7fbb8fe00640 1 -- 172.21.15.49:0/2830909743 learned_addr learned my addr 172.21.15.49:0/2830909743 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:42.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.410+0000 7fbb8f400640 1 --2- 172.21.15.49:0/2830909743 >> v2:172.21.15.107:3300/0 conn(0x7fbb900cee00 0x7fbb9006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.410+0000 7fbb8e000640 1 -- 172.21.15.49:0/2830909743 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531125729 0 0) 0x7fbb9015dbb0 con 0x7fbb9015d2f0 2024-04-15T15:18:42.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.410+0000 7fbb8e000640 1 -- 172.21.15.49:0/2830909743 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb74003520 con 0x7fbb9015d2f0 2024-04-15T15:18:42.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.411+0000 7fbb8f400640 1 -- 172.21.15.49:0/2830909743 >> v1:172.21.15.107:6789/0 conn(0x7fbb90071f50 legacy=0x7fbb9015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.411+0000 7fbb8f400640 1 -- 172.21.15.49:0/2830909743 >> v1:172.21.15.49:6789/0 conn(0x7fbb9015d2f0 legacy=0x7fbb9015dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.411+0000 7fbb8f400640 1 -- 172.21.15.49:0/2830909743 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9006dde0 con 0x7fbb900cee00 2024-04-15T15:18:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.412+0000 7fbb8f400640 1 --2- 172.21.15.49:0/2830909743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb900cee00 0x7fbb9006e8c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbb800097a0 tx=0x7fbb8002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b03f73456a6b6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.412+0000 7fbb8e000640 1 -- 172.21.15.49:0/2830909743 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80034070 con 0x7fbb900cee00 2024-04-15T15:18:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.412+0000 7fbb8e000640 1 -- 172.21.15.49:0/2830909743 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb80030610 con 0x7fbb900cee00 2024-04-15T15:18:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.412+0000 7fbb8e000640 1 -- 172.21.15.49:0/2830909743 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb800309e0 con 0x7fbb900cee00 2024-04-15T15:18:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.413+0000 7fbb95000640 1 -- 172.21.15.49:0/2830909743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb900cee00 msgr2=0x7fbb9006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.413+0000 7fbb95000640 1 --2- 172.21.15.49:0/2830909743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb900cee00 0x7fbb9006e8c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbb800097a0 tx=0x7fbb8002b960 comp rx=0 tx=0).stop 2024-04-15T15:18:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.413+0000 7fbb95000640 1 -- 172.21.15.49:0/2830909743 shutdown_connections 2024-04-15T15:18:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.413+0000 7fbb95000640 1 --2- 172.21.15.49:0/2830909743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb900cee00 0x7fbb9006e8c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.413+0000 7fbb95000640 1 -- 172.21.15.49:0/2830909743 >> 172.21.15.49:0/2830909743 conn(0x7fbb9006b930 msgr2=0x7fbb9006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.413+0000 7fbb95000640 1 -- 172.21.15.49:0/2830909743 shutdown_connections 2024-04-15T15:18:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.413+0000 7fbb95000640 1 -- 172.21.15.49:0/2830909743 wait complete. 2024-04-15T15:18:42.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.415+0000 7fbb95000640 1 Processor -- start 2024-04-15T15:18:42.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.415+0000 7fbb95000640 1 -- start start 2024-04-15T15:18:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.416+0000 7fbb95000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90071f50 0x7fbb901e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.416+0000 7fbb95000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb900cee00 0x7fbb901f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.416+0000 7fbb95000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb9015d2f0 0x7fbb9015eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.416+0000 7fbb95000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb900723b0 con 0x7fbb90071f50 2024-04-15T15:18:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.416+0000 7fbb95000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9006e3a0 con 0x7fbb900cee00 2024-04-15T15:18:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.416+0000 7fbb95000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fbb90072530 con 0x7fbb9015d2f0 2024-04-15T15:18:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.416+0000 7fbb8f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90071f50 0x7fbb901e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.416+0000 7fbb8f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90071f50 0x7fbb901e6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45056/0 (socket says 172.21.15.49:45056) 2024-04-15T15:18:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.416+0000 7fbb8f400640 1 -- 172.21.15.49:0/2475866474 learned_addr learned my addr 172.21.15.49:0/2475866474 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.416+0000 7fbb8ea00640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb900cee00 0x7fbb901f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb8fe00640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb9015d2f0 0x7fbb9015eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb8f400640 1 -- 172.21.15.49:0/2475866474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb9015d2f0 msgr2=0x7fbb9015eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb8f400640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb9015d2f0 0x7fbb9015eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb8f400640 1 -- 172.21.15.49:0/2475866474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb900cee00 msgr2=0x7fbb901f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb8f400640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb900cee00 0x7fbb901f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb8f400640 1 -- 172.21.15.49:0/2475866474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb90147e70 con 0x7fbb90071f50 2024-04-15T15:18:42.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb8f400640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90071f50 0x7fbb901e6a20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb80000c00 tx=0x7fbb800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:42.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb87400640 1 -- 172.21.15.49:0/2475866474 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80034040 con 0x7fbb90071f50 2024-04-15T15:18:42.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb87400640 1 -- 172.21.15.49:0/2475866474 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb80030410 con 0x7fbb90071f50 2024-04-15T15:18:42.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb95000640 1 -- 172.21.15.49:0/2475866474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb90148150 con 0x7fbb90071f50 2024-04-15T15:18:42.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb87400640 1 -- 172.21.15.49:0/2475866474 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80004250 con 0x7fbb90071f50 2024-04-15T15:18:42.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.417+0000 7fbb95000640 1 -- 172.21.15.49:0/2475866474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb90148710 con 0x7fbb90071f50 2024-04-15T15:18:42.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.419+0000 7fbb95000640 1 -- 172.21.15.49:0/2475866474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb900755c0 con 0x7fbb90071f50 2024-04-15T15:18:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.419+0000 7fbb87400640 1 -- 172.21.15.49:0/2475866474 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fbb8004c060 con 0x7fbb90071f50 2024-04-15T15:18:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.420+0000 7fbb87400640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbb7006c890 0x7fbb7006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.420+0000 7fbb87400640 1 -- 172.21.15.49:0/2475866474 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fbb800ad150 con 0x7fbb90071f50 2024-04-15T15:18:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.421+0000 7fbb8ea00640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbb7006c890 0x7fbb7006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.421+0000 7fbb8ea00640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbb7006c890 0x7fbb7006ed50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbb901f7970 tx=0x7fbb7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:42.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.425+0000 7fbb87400640 1 -- 172.21.15.49:0/2475866474 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb8007b920 con 0x7fbb90071f50 2024-04-15T15:18:42.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.642+0000 7fbb95000640 1 -- 172.21.15.49:0/2475866474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbb9015c810 con 0x7fbb90071f50 2024-04-15T15:18:42.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.642+0000 7fbb87400640 1 -- 172.21.15.49:0/2475866474 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fbb8007b220 con 0x7fbb90071f50 2024-04-15T15:18:42.644 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:42.644 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2024-04-15T15:18:42.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:18:42.644 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2024-04-15T15:18:42.644 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2024-04-15T15:18:42.645 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:18:42.645 INFO:teuthology.orchestra.run.smithi049.stdout:] 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 -- 172.21.15.49:0/2475866474 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbb7006c890 msgr2=0x7fbb7006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbb7006c890 0x7fbb7006ed50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbb901f7970 tx=0x7fbb7c009040 comp rx=0 tx=0).stop 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 -- 172.21.15.49:0/2475866474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90071f50 msgr2=0x7fbb901e6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90071f50 0x7fbb901e6a20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb80000c00 tx=0x7fbb800046c0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 -- 172.21.15.49:0/2475866474 shutdown_connections 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbb7006c890 0x7fbb7006ed50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb9015d2f0 0x7fbb9015eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb900cee00 0x7fbb901f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 --2- 172.21.15.49:0/2475866474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90071f50 0x7fbb901e6a20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 -- 172.21.15.49:0/2475866474 >> 172.21.15.49:0/2475866474 conn(0x7fbb9006b930 msgr2=0x7fbb900b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 -- 172.21.15.49:0/2475866474 shutdown_connections 2024-04-15T15:18:42.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.645+0000 7fbb95000640 1 -- 172.21.15.49:0/2475866474 wait complete. 2024-04-15T15:18:42.663 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-15T15:18:42.664 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-15T15:18:42.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.885+0000 7febb6e00640 1 Processor -- start 2024-04-15T15:18:42.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.885+0000 7febb6e00640 1 -- start start 2024-04-15T15:18:42.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.887+0000 7febb6e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7febb0072860 0x7febb00cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.887+0000 7febb6e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7febb00cd680 0x7febb015d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.887+0000 7febb6e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb00ce000 con 0x7febb01462f0 2024-04-15T15:18:42.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.887+0000 7febb6e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7febb00a9640 con 0x7febb0072860 2024-04-15T15:18:42.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.887+0000 7febb6e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7febb00cdd20 con 0x7febb00cd680 2024-04-15T15:18:42.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.887+0000 7febafe00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7febb01462f0 0x7febb0072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48850/0 (socket says 172.21.15.49:48850) 2024-04-15T15:18:42.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.887+0000 7febafe00640 1 -- 172.21.15.49:0/3942239717 learned_addr learned my addr 172.21.15.49:0/3942239717 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:42.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.887+0000 7feba7400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7febb0072860 0x7febb00cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.888+0000 7febb4e00640 1 --2- 172.21.15.49:0/3942239717 >> v2:172.21.15.107:3300/0 conn(0x7febb00cd680 0x7febb015d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.888+0000 7feba7400640 1 -- 172.21.15.49:0/3942239717 >> v2:172.21.15.107:3300/0 conn(0x7febb00cd680 msgr2=0x7febb015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.888+0000 7feba7400640 1 --2- 172.21.15.49:0/3942239717 >> v2:172.21.15.107:3300/0 conn(0x7febb00cd680 0x7febb015d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.888+0000 7feba7400640 1 -- 172.21.15.49:0/3942239717 >> v1:172.21.15.49:6789/0 conn(0x7febb01462f0 legacy=0x7febb0072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.888+0000 7feba7400640 1 -- 172.21.15.49:0/3942239717 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb015cf10 con 0x7febb0072860 2024-04-15T15:18:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.889+0000 7feba7400640 1 --2- 172.21.15.49:0/3942239717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb0072860 0x7febb00cd140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7febb00cd0d0 tx=0x7feb9802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441f8713533d10f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.889+0000 7febaf400640 1 -- 172.21.15.49:0/3942239717 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98034070 con 0x7febb0072860 2024-04-15T15:18:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.889+0000 7febaf400640 1 -- 172.21.15.49:0/3942239717 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb98030610 con 0x7febb0072860 2024-04-15T15:18:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.889+0000 7febaf400640 1 -- 172.21.15.49:0/3942239717 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980309e0 con 0x7febb0072860 2024-04-15T15:18:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.890+0000 7febb6e00640 1 -- 172.21.15.49:0/3942239717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb0072860 msgr2=0x7febb00cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.890+0000 7febb6e00640 1 --2- 172.21.15.49:0/3942239717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb0072860 0x7febb00cd140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7febb00cd0d0 tx=0x7feb9802b960 comp rx=0 tx=0).stop 2024-04-15T15:18:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.890+0000 7febb6e00640 1 -- 172.21.15.49:0/3942239717 shutdown_connections 2024-04-15T15:18:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.890+0000 7febb6e00640 1 --2- 172.21.15.49:0/3942239717 >> v2:172.21.15.107:3300/0 conn(0x7febb00cd680 0x7febb015d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.890+0000 7febb6e00640 1 --2- 172.21.15.49:0/3942239717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb0072860 0x7febb00cd140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.890+0000 7febb6e00640 1 -- 172.21.15.49:0/3942239717 >> 172.21.15.49:0/3942239717 conn(0x7febb00b2460 msgr2=0x7febb00b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.890+0000 7febb6e00640 1 -- 172.21.15.49:0/3942239717 shutdown_connections 2024-04-15T15:18:42.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.890+0000 7febb6e00640 1 -- 172.21.15.49:0/3942239717 wait complete. 2024-04-15T15:18:42.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.892+0000 7febb6e00640 1 Processor -- start 2024-04-15T15:18:42.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.892+0000 7febb6e00640 1 -- start start 2024-04-15T15:18:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.893+0000 7febb6e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febb0072860 0x7febb01e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.893+0000 7febb6e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb00cd680 0x7febb013fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.893+0000 7febb6e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb01462f0 0x7febb013d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.893+0000 7febb6e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7febb015df30 con 0x7febb01462f0 2024-04-15T15:18:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.893+0000 7febb6e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7febb015d4d0 con 0x7febb00cd680 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.893+0000 7febb6e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7febb015e0b0 con 0x7febb0072860 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.893+0000 7febb4e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb01462f0 0x7febb013d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.893+0000 7febb4e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb01462f0 0x7febb013d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45066/0 (socket says 172.21.15.49:45066) 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.893+0000 7febb4e00640 1 -- 172.21.15.49:0/1542636293 learned_addr learned my addr 172.21.15.49:0/1542636293 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.893+0000 7feba7400640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb00cd680 0x7febb013fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.894+0000 7febafe00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febb0072860 0x7febb01e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.894+0000 7febb4e00640 1 -- 172.21.15.49:0/1542636293 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febb0072860 msgr2=0x7febb01e4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.894+0000 7febb4e00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febb0072860 0x7febb01e4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.894+0000 7febb4e00640 1 -- 172.21.15.49:0/1542636293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb00cd680 msgr2=0x7febb013fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.894+0000 7febb4e00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb00cd680 0x7febb013fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.894+0000 7febb4e00640 1 -- 172.21.15.49:0/1542636293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb0140440 con 0x7febb01462f0 2024-04-15T15:18:42.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.894+0000 7febb4e00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb01462f0 0x7febb013d5b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feba800d800 tx=0x7feba800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:42.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.894+0000 7febacc00640 1 -- 172.21.15.49:0/1542636293 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba800b6c0 con 0x7febb01462f0 2024-04-15T15:18:42.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.895+0000 7febacc00640 1 -- 172.21.15.49:0/1542636293 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba8019070 con 0x7febb01462f0 2024-04-15T15:18:42.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.895+0000 7febacc00640 1 -- 172.21.15.49:0/1542636293 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba800bac0 con 0x7febb01462f0 2024-04-15T15:18:42.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.895+0000 7febb6e00640 1 -- 172.21.15.49:0/1542636293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb0142620 con 0x7febb01462f0 2024-04-15T15:18:42.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.895+0000 7febb6e00640 1 -- 172.21.15.49:0/1542636293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb013daf0 con 0x7febb01462f0 2024-04-15T15:18:42.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.896+0000 7febb6e00640 1 -- 172.21.15.49:0/1542636293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb015c7a0 con 0x7febb01462f0 2024-04-15T15:18:42.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.897+0000 7febacc00640 1 -- 172.21.15.49:0/1542636293 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7feba800bc60 con 0x7febb01462f0 2024-04-15T15:18:42.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.898+0000 7febacc00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feb8c06c860 0x7feb8c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:42.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.898+0000 7febacc00640 1 -- 172.21.15.49:0/1542636293 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7feba808bb90 con 0x7febb01462f0 2024-04-15T15:18:42.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.898+0000 7febafe00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feb8c06c860 0x7feb8c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:42.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.899+0000 7febafe00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feb8c06c860 0x7feb8c06ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7febb01442c0 tx=0x7feba0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:42.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:42.903+0000 7febacc00640 1 -- 172.21.15.49:0/1542636293 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febb015c7a0 con 0x7febb01462f0 2024-04-15T15:18:43.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.107+0000 7febb6e00640 1 -- 172.21.15.49:0/1542636293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7febb0068db0 con 0x7febb01462f0 2024-04-15T15:18:43.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.816+0000 7febacc00640 1 -- 172.21.15.49:0/1542636293 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7feba805a350 con 0x7febb01462f0 2024-04-15T15:18:43.818 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2024-04-15T15:18:43.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.820+0000 7febb6e00640 1 -- 172.21.15.49:0/1542636293 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feb8c06c860 msgr2=0x7feb8c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:43.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.820+0000 7febb6e00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feb8c06c860 0x7feb8c06ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7febb01442c0 tx=0x7feba0009040 comp rx=0 tx=0).stop 2024-04-15T15:18:43.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.820+0000 7febb6e00640 1 -- 172.21.15.49:0/1542636293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb01462f0 msgr2=0x7febb013d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:43.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.820+0000 7febb6e00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb01462f0 0x7febb013d5b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feba800d800 tx=0x7feba800dcc0 comp rx=0 tx=0).stop 2024-04-15T15:18:43.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.820+0000 7febb6e00640 1 -- 172.21.15.49:0/1542636293 shutdown_connections 2024-04-15T15:18:43.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.820+0000 7febb6e00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feb8c06c860 0x7feb8c06ed20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:43.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.820+0000 7febb6e00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febb01462f0 0x7febb013d5b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:43.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.821+0000 7febb6e00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb00cd680 0x7febb013fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:43.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.821+0000 7febb6e00640 1 --2- 172.21.15.49:0/1542636293 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febb0072860 0x7febb01e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:43.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.821+0000 7febb6e00640 1 -- 172.21.15.49:0/1542636293 >> 172.21.15.49:0/1542636293 conn(0x7febb00b2460 msgr2=0x7febb00ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:43.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.821+0000 7febb6e00640 1 -- 172.21.15.49:0/1542636293 shutdown_connections 2024-04-15T15:18:43.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:43.821+0000 7febb6e00640 1 -- 172.21.15.49:0/1542636293 wait complete. 2024-04-15T15:18:43.855 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-15T15:18:44.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.065+0000 7fe651800640 1 Processor -- start 2024-04-15T15:18:44.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.065+0000 7fe651800640 1 -- start start 2024-04-15T15:18:44.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.066+0000 7fe651800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe64c0b3c70 0x7fe64c0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:44.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.067+0000 7fe651800640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe64c0ba4d0 con 0x7fe64c0b3720 2024-04-15T15:18:44.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.067+0000 7fe651800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe64c0ba7b0 con 0x7fe64c15ecc0 2024-04-15T15:18:44.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.067+0000 7fe651800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe64c15f0a0 con 0x7fe64c0b3c70 2024-04-15T15:18:44.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.067+0000 7fe64a000640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe64c0b3720 0x7fe64c0b95f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43978/0 (socket says 172.21.15.49:43978) 2024-04-15T15:18:44.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.067+0000 7fe64a000640 1 -- 172.21.15.49:0/2884532759 learned_addr learned my addr 172.21.15.49:0/2884532759 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:44.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.067+0000 7fe64b400640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe64c0b3c70 0x7fe64c0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:44.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037705105 0 0) 0x7fe64c0ba4d0 con 0x7fe64c0b3720 2024-04-15T15:18:44.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe630003520 con 0x7fe64c0b3720 2024-04-15T15:18:44.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1522713033 0 0) 0x7fe630003520 con 0x7fe64c0b3720 2024-04-15T15:18:44.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe64c0b3c70 msgr2=0x7fe64c0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:44.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe64c0b3c70 0x7fe64c0cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:44.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 >> v1:172.21.15.107:6789/0 conn(0x7fe64c15ecc0 legacy=0x7fe64c0d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:44.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c0b8df0 con 0x7fe64c0b3720 2024-04-15T15:18:44.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fe638001670 con 0x7fe64c0b3720 2024-04-15T15:18:44.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 >> v1:172.21.15.49:6789/0 conn(0x7fe64c0b3720 legacy=0x7fe64c0b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:44.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe64c0b3c70 0x7fe6300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:44.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe630003f20 0x7fe630004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:44.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630004d40 0x7fe6300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:44.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c15f0a0 con 0x7fe630003f20 2024-04-15T15:18:44.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe63000ba70 con 0x7fe630004d40 2024-04-15T15:18:44.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.068+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe63000bc70 con 0x7fe64c0b3c70 2024-04-15T15:18:44.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.069+0000 7fe64a000640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe630003f20 0x7fe630004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:44.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.069+0000 7fe64aa00640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe64c0b3c70 0x7fe6300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:44.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.069+0000 7fe64b400640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630004d40 0x7fe6300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:44.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.069+0000 7fe64a000640 1 -- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe64c0b3c70 msgr2=0x7fe6300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:44.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.069+0000 7fe64a000640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe64c0b3c70 0x7fe6300039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:44.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.069+0000 7fe64a000640 1 -- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630004d40 msgr2=0x7fe6300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:44.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.069+0000 7fe64a000640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe630004d40 0x7fe6300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:44.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.069+0000 7fe64a000640 1 -- 172.21.15.49:0/2884532759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe630003520 con 0x7fe630003f20 2024-04-15T15:18:44.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.070+0000 7fe64a000640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe630003f20 0x7fe630004800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c190 tx=0x7fe630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0aa627e41e42cb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:44.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.071+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638007c60 con 0x7fe630003f20 2024-04-15T15:18:44.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.071+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe638030070 con 0x7fe630003f20 2024-04-15T15:18:44.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.071+0000 7fe649600640 1 -- 172.21.15.49:0/2884532759 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638005d10 con 0x7fe630003f20 2024-04-15T15:18:44.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.071+0000 7fe651800640 1 -- 172.21.15.49:0/2884532759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe630003f20 msgr2=0x7fe630004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:44.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.071+0000 7fe651800640 1 --2- 172.21.15.49:0/2884532759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe630003f20 0x7fe630004800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c190 tx=0x7fe630005830 comp rx=0 tx=0).stop 2024-04-15T15:18:44.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.071+0000 7fe64b400640 1 -- 172.21.15.49:0/2884532759 reap_dead start 2024-04-15T15:18:44.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.071+0000 7fe651800640 1 -- 172.21.15.49:0/2884532759 shutdown_connections 2024-04-15T15:18:44.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.071+0000 7fe651800640 1 -- 172.21.15.49:0/2884532759 >> 172.21.15.49:0/2884532759 conn(0x7fe64c073480 msgr2=0x7fe64c073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:44.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.071+0000 7fe651800640 1 -- 172.21.15.49:0/2884532759 shutdown_connections 2024-04-15T15:18:44.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.072+0000 7fe651800640 1 -- 172.21.15.49:0/2884532759 wait complete. 2024-04-15T15:18:44.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.073+0000 7fe651800640 1 Processor -- start 2024-04-15T15:18:44.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.074+0000 7fe651800640 1 -- start start 2024-04-15T15:18:44.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.074+0000 7fe651800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe630003f20 0x7fe64c14c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:44.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.074+0000 7fe651800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe64c0cba30 0x7fe64c0cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:44.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe651800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64c0cb050 0x7fe64c0d4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:44.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe651800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c0b8aa0 con 0x7fe64c0cb050 2024-04-15T15:18:44.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe651800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c0ba280 con 0x7fe64c0cba30 2024-04-15T15:18:44.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe651800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe64c0d3440 con 0x7fe630003f20 2024-04-15T15:18:44.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe64b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64c0cb050 0x7fe64c0d4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:44.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe64b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64c0cb050 0x7fe64c0d4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58166/0 (socket says 172.21.15.49:58166) 2024-04-15T15:18:44.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe64b400640 1 -- 172.21.15.49:0/228199243 learned_addr learned my addr 172.21.15.49:0/228199243 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:44.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe64aa00640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe630003f20 0x7fe64c14c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:44.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe64a000640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe64c0cba30 0x7fe64c0cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:44.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe64b400640 1 -- 172.21.15.49:0/228199243 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe630003f20 msgr2=0x7fe64c14c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:44.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe64b400640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe630003f20 0x7fe64c14c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:44.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe64b400640 1 -- 172.21.15.49:0/228199243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe64c0cba30 msgr2=0x7fe64c0cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:44.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe64b400640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe64c0cba30 0x7fe64c0cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:44.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.075+0000 7fe64b400640 1 -- 172.21.15.49:0/228199243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c1605f0 con 0x7fe64c0cb050 2024-04-15T15:18:44.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.076+0000 7fe64b400640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64c0cb050 0x7fe64c0d4cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0c8300 tx=0x7fe6440115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:44.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.076+0000 7fe642a00640 1 -- 172.21.15.49:0/228199243 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644011e10 con 0x7fe64c0cb050 2024-04-15T15:18:44.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.076+0000 7fe642a00640 1 -- 172.21.15.49:0/228199243 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe64401b070 con 0x7fe64c0cb050 2024-04-15T15:18:44.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.076+0000 7fe642a00640 1 -- 172.21.15.49:0/228199243 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644003990 con 0x7fe64c0cb050 2024-04-15T15:18:44.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.076+0000 7fe651800640 1 -- 172.21.15.49:0/228199243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c1608d0 con 0x7fe64c0cb050 2024-04-15T15:18:44.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.076+0000 7fe651800640 1 -- 172.21.15.49:0/228199243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c160e90 con 0x7fe64c0cb050 2024-04-15T15:18:44.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.077+0000 7fe651800640 1 -- 172.21.15.49:0/228199243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c069320 con 0x7fe64c0cb050 2024-04-15T15:18:44.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.078+0000 7fe642a00640 1 -- 172.21.15.49:0/228199243 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe644005d10 con 0x7fe64c0cb050 2024-04-15T15:18:44.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.079+0000 7fe642a00640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe62c06c8f0 0x7fe62c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:44.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.079+0000 7fe642a00640 1 -- 172.21.15.49:0/228199243 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7fe64408bfa0 con 0x7fe64c0cb050 2024-04-15T15:18:44.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.079+0000 7fe64aa00640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe62c06c8f0 0x7fe62c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:44.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.080+0000 7fe64aa00640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe62c06c8f0 0x7fe62c06edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe63c005540 tx=0x7fe63c007c50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:44.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.085+0000 7fe642a00640 1 -- 172.21.15.49:0/228199243 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe64c069320 con 0x7fe64c0cb050 2024-04-15T15:18:44.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.294+0000 7fe651800640 1 -- 172.21.15.49:0/228199243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe64c068c80 con 0x7fe64c0cb050 2024-04-15T15:18:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.827+0000 7fe642a00640 1 -- 172.21.15.49:0/228199243 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7fe644025020 con 0x7fe64c0cb050 2024-04-15T15:18:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2024-04-15T15:18:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.831+0000 7fe651800640 1 -- 172.21.15.49:0/228199243 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe62c06c8f0 msgr2=0x7fe62c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.831+0000 7fe651800640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe62c06c8f0 0x7fe62c06edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe63c005540 tx=0x7fe63c007c50 comp rx=0 tx=0).stop 2024-04-15T15:18:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.831+0000 7fe651800640 1 -- 172.21.15.49:0/228199243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64c0cb050 msgr2=0x7fe64c0d4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.831+0000 7fe651800640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64c0cb050 0x7fe64c0d4cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0c8300 tx=0x7fe6440115a0 comp rx=0 tx=0).stop 2024-04-15T15:18:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.831+0000 7fe651800640 1 -- 172.21.15.49:0/228199243 shutdown_connections 2024-04-15T15:18:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.831+0000 7fe651800640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe62c06c8f0 0x7fe62c06edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.831+0000 7fe651800640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64c0cb050 0x7fe64c0d4cf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.831+0000 7fe651800640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe64c0cba30 0x7fe64c0cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.831+0000 7fe651800640 1 --2- 172.21.15.49:0/228199243 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe630003f20 0x7fe64c14c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.831+0000 7fe651800640 1 -- 172.21.15.49:0/228199243 >> 172.21.15.49:0/228199243 conn(0x7fe64c073480 msgr2=0x7fe64c15c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:44.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.832+0000 7fe651800640 1 -- 172.21.15.49:0/228199243 shutdown_connections 2024-04-15T15:18:44.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:44.832+0000 7fe651800640 1 -- 172.21.15.49:0/228199243 wait complete. 2024-04-15T15:18:44.872 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-15T15:18:45.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.063+0000 7f8225200640 1 Processor -- start 2024-04-15T15:18:45.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.063+0000 7f8225200640 1 -- start start 2024-04-15T15:18:45.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.065+0000 7f8225200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f822006e2f0 0x7f82200d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.065+0000 7f8225200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8220071330 con 0x7f82200ce650 2024-04-15T15:18:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.065+0000 7f8225200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f822006cae0 con 0x7f822009b230 2024-04-15T15:18:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.065+0000 7f8225200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f822009bb40 con 0x7f822006e2f0 2024-04-15T15:18:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.065+0000 7f821de00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f82200ce650 0x7f822009ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43982/0 (socket says 172.21.15.49:43982) 2024-04-15T15:18:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.065+0000 7f821de00640 1 -- 172.21.15.49:0/3939572540 learned_addr learned my addr 172.21.15.49:0/3939572540 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 998157694 0 0) 0x7f8220071330 con 0x7f82200ce650 2024-04-15T15:18:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8204003520 con 0x7f82200ce650 2024-04-15T15:18:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821e800640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f822006e2f0 0x7f82200d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1637981681 0 0) 0x7f8204003520 con 0x7f82200ce650 2024-04-15T15:18:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f822006e2f0 msgr2=0x7f82200d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f822006e2f0 0x7f82200d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 >> v1:172.21.15.107:6789/0 conn(0x7f822009b230 legacy=0x7f82200d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822006e9f0 con 0x7f82200ce650 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f8210001670 con 0x7f82200ce650 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 >> v1:172.21.15.49:6789/0 conn(0x7f82200ce650 legacy=0x7f822009ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82040039e0 0x7f8204003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8204004800 0x7f8204004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f822006e2f0 0x7f8204005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f822009bb40 con 0x7f8204004800 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f820400bb80 con 0x7f82040039e0 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.066+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f820400bd80 con 0x7f822006e2f0 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.067+0000 7f8215400640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8204004800 0x7f8204004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.067+0000 7f821e800640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f822006e2f0 0x7f8204005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.067+0000 7f821de00640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82040039e0 0x7f8204003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.067+0000 7f8215400640 1 -- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f822006e2f0 msgr2=0x7f8204005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.067+0000 7f8215400640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f822006e2f0 0x7f8204005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.067+0000 7f8215400640 1 -- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82040039e0 msgr2=0x7f8204003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.067+0000 7f8215400640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82040039e0 0x7f8204003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.067+0000 7f8215400640 1 -- 172.21.15.49:0/3939572540 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8204003520 con 0x7f8204004800 2024-04-15T15:18:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.068+0000 7f8215400640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8204004800 0x7f8204004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8208001750 tx=0x7f8204005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e7834fe31ef0c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.068+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208001370 con 0x7f8204004800 2024-04-15T15:18:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.068+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82080052d0 con 0x7f8204004800 2024-04-15T15:18:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.068+0000 7f821d400640 1 -- 172.21.15.49:0/3939572540 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208003830 con 0x7f8204004800 2024-04-15T15:18:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.069+0000 7f8225200640 1 -- 172.21.15.49:0/3939572540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8204004800 msgr2=0x7f8204004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.069+0000 7f8225200640 1 --2- 172.21.15.49:0/3939572540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8204004800 0x7f8204004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8208001750 tx=0x7f8204005830 comp rx=0 tx=0).stop 2024-04-15T15:18:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.069+0000 7f821e800640 1 -- 172.21.15.49:0/3939572540 reap_dead start 2024-04-15T15:18:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.069+0000 7f8225200640 1 -- 172.21.15.49:0/3939572540 shutdown_connections 2024-04-15T15:18:45.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.069+0000 7f8225200640 1 -- 172.21.15.49:0/3939572540 >> 172.21.15.49:0/3939572540 conn(0x7f82200d2720 msgr2=0x7f82200d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:45.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.069+0000 7f8225200640 1 -- 172.21.15.49:0/3939572540 shutdown_connections 2024-04-15T15:18:45.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.069+0000 7f8225200640 1 -- 172.21.15.49:0/3939572540 wait complete. 2024-04-15T15:18:45.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.071+0000 7f8225200640 1 Processor -- start 2024-04-15T15:18:45.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.071+0000 7f8225200640 1 -- start start 2024-04-15T15:18:45.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.072+0000 7f8225200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8204004800 0x7f82200c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:45.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.072+0000 7f8225200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82200b6ac0 0x7f82201f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:45.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.072+0000 7f8225200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f822015d000 0x7f822015d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:45.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.072+0000 7f8225200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f822009bd90 con 0x7f82200b6ac0 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.072+0000 7f8225200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f822006ce50 con 0x7f822015d000 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.072+0000 7f8225200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f82200cea30 con 0x7f8204004800 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.072+0000 7f8215400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82200b6ac0 0x7f82201f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.072+0000 7f8215400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82200b6ac0 0x7f82201f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58192/0 (socket says 172.21.15.49:58192) 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.072+0000 7f8215400640 1 -- 172.21.15.49:0/135990677 learned_addr learned my addr 172.21.15.49:0/135990677 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.073+0000 7f821e800640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f822015d000 0x7f822015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.073+0000 7f821de00640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8204004800 0x7f82200c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.073+0000 7f8215400640 1 -- 172.21.15.49:0/135990677 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8204004800 msgr2=0x7f82200c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.073+0000 7f8215400640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8204004800 0x7f82200c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.073+0000 7f8215400640 1 -- 172.21.15.49:0/135990677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f822015d000 msgr2=0x7f822015d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.073+0000 7f8215400640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f822015d000 0x7f822015d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.073+0000 7f8215400640 1 -- 172.21.15.49:0/135990677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82200b59a0 con 0x7f82200b6ac0 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.073+0000 7f8215400640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82200b6ac0 0x7f82201f3f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8208005040 tx=0x7f8208001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:45.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.073+0000 7f8216a00640 1 -- 172.21.15.49:0/135990677 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208041070 con 0x7f82200b6ac0 2024-04-15T15:18:45.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.074+0000 7f8216a00640 1 -- 172.21.15.49:0/135990677 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8208005480 con 0x7f82200b6ac0 2024-04-15T15:18:45.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.074+0000 7f8216a00640 1 -- 172.21.15.49:0/135990677 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208001370 con 0x7f82200b6ac0 2024-04-15T15:18:45.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.074+0000 7f8225200640 1 -- 172.21.15.49:0/135990677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82200c9e40 con 0x7f82200b6ac0 2024-04-15T15:18:45.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.074+0000 7f8225200640 1 -- 172.21.15.49:0/135990677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82200ca400 con 0x7f82200b6ac0 2024-04-15T15:18:45.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.075+0000 7f8225200640 1 -- 172.21.15.49:0/135990677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8220068c80 con 0x7f82200b6ac0 2024-04-15T15:18:45.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.076+0000 7f8216a00640 1 -- 172.21.15.49:0/135990677 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8208044070 con 0x7f82200b6ac0 2024-04-15T15:18:45.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.077+0000 7f8216a00640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81fc06c920 0x7f81fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.077+0000 7f8216a00640 1 -- 172.21.15.49:0/135990677 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f82080ad6d0 con 0x7f82200b6ac0 2024-04-15T15:18:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.077+0000 7f821de00640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81fc06c920 0x7f81fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.078+0000 7f821de00640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81fc06c920 0x7f81fc06ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8210001050 tx=0x7f8210009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:45.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.082+0000 7f8216a00640 1 -- 172.21.15.49:0/135990677 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82200b59a0 con 0x7f82200b6ac0 2024-04-15T15:18:45.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.324+0000 7f8225200640 1 -- 172.21.15.49:0/135990677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f82200d3620 con 0x7f82200b6ac0 2024-04-15T15:18:45.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.845+0000 7f8216a00640 1 -- 172.21.15.49:0/135990677 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f820807bc70 con 0x7f82200b6ac0 2024-04-15T15:18:45.847 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 2 and data pool 3 2024-04-15T15:18:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.853+0000 7f8225200640 1 -- 172.21.15.49:0/135990677 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81fc06c920 msgr2=0x7f81fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.853+0000 7f8225200640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81fc06c920 0x7f81fc06ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8210001050 tx=0x7f8210009040 comp rx=0 tx=0).stop 2024-04-15T15:18:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.853+0000 7f8225200640 1 -- 172.21.15.49:0/135990677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82200b6ac0 msgr2=0x7f82201f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.853+0000 7f8225200640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82200b6ac0 0x7f82201f3f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8208005040 tx=0x7f8208001120 comp rx=0 tx=0).stop 2024-04-15T15:18:45.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.854+0000 7f8225200640 1 -- 172.21.15.49:0/135990677 shutdown_connections 2024-04-15T15:18:45.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.854+0000 7f8225200640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81fc06c920 0x7f81fc06ede0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:45.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.854+0000 7f8225200640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f822015d000 0x7f822015d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:45.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.854+0000 7f8225200640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82200b6ac0 0x7f82201f3f40 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8208005040 tx=0x7f8208001120 comp rx=0 tx=0).stop 2024-04-15T15:18:45.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.854+0000 7f8225200640 1 --2- 172.21.15.49:0/135990677 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8204004800 0x7f82200c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:45.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.854+0000 7f8225200640 1 -- 172.21.15.49:0/135990677 >> 172.21.15.49:0/135990677 conn(0x7f82200d2720 msgr2=0x7f82200b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:45.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.856+0000 7f8225200640 1 -- 172.21.15.49:0/135990677 shutdown_connections 2024-04-15T15:18:45.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:45.856+0000 7f8225200640 1 -- 172.21.15.49:0/135990677 wait complete. 2024-04-15T15:18:45.884 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:46.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.069+0000 7f89c2200640 1 Processor -- start 2024-04-15T15:18:46.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.069+0000 7f89c2200640 1 -- start start 2024-04-15T15:18:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.071+0000 7f89c2200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f89bc0cee00 0x7f89bc06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.071+0000 7f89c2200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f89bc071f50 0x7f89bc0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.071+0000 7f89c2200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89bc15dbb0 con 0x7f89bc15cbe0 2024-04-15T15:18:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.071+0000 7f89c2200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f89bc06f4f0 con 0x7f89bc0cee00 2024-04-15T15:18:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.071+0000 7f89c2200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f89bc0728f0 con 0x7f89bc071f50 2024-04-15T15:18:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.071+0000 7f89c1800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f89bc15cbe0 0x7f89bc15d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43984/0 (socket says 172.21.15.49:43984) 2024-04-15T15:18:46.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.071+0000 7f89c1800640 1 -- 172.21.15.49:0/4134069649 learned_addr learned my addr 172.21.15.49:0/4134069649 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.071+0000 7f89c0e00640 1 --2- 172.21.15.49:0/4134069649 >> v2:172.21.15.49:3300/0 conn(0x7f89bc0cee00 0x7f89bc06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.071+0000 7f89bbe00640 1 --2- 172.21.15.49:0/4134069649 >> v2:172.21.15.107:3300/0 conn(0x7f89bc071f50 0x7f89bc0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.072+0000 7f89bb400640 1 -- 172.21.15.49:0/4134069649 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2086384625 0 0) 0x7f89bc15dbb0 con 0x7f89bc15cbe0 2024-04-15T15:18:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.072+0000 7f89bb400640 1 -- 172.21.15.49:0/4134069649 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89a4003520 con 0x7f89bc15cbe0 2024-04-15T15:18:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.072+0000 7f89c0e00640 1 -- 172.21.15.49:0/4134069649 >> v2:172.21.15.107:3300/0 conn(0x7f89bc071f50 msgr2=0x7f89bc0723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.072+0000 7f89c0e00640 1 --2- 172.21.15.49:0/4134069649 >> v2:172.21.15.107:3300/0 conn(0x7f89bc071f50 0x7f89bc0723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.072+0000 7f89c0e00640 1 -- 172.21.15.49:0/4134069649 >> v1:172.21.15.49:6789/0 conn(0x7f89bc15cbe0 legacy=0x7f89bc15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.072+0000 7f89c0e00640 1 -- 172.21.15.49:0/4134069649 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc15d8d0 con 0x7f89bc0cee00 2024-04-15T15:18:46.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.073+0000 7f89c0e00640 1 --2- 172.21.15.49:0/4134069649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 0x7f89bc06e8c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0b5aa0 tx=0x7f89b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2faa3ea76546871c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:46.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.073+0000 7f89bb400640 1 -- 172.21.15.49:0/4134069649 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0035020 con 0x7f89bc0cee00 2024-04-15T15:18:46.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.073+0000 7f89bb400640 1 -- 172.21.15.49:0/4134069649 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b0031830 con 0x7f89bc0cee00 2024-04-15T15:18:46.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.073+0000 7f89bb400640 1 -- 172.21.15.49:0/4134069649 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0031c00 con 0x7f89bc0cee00 2024-04-15T15:18:46.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.074+0000 7f89c2200640 1 -- 172.21.15.49:0/4134069649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 msgr2=0x7f89bc06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.074+0000 7f89c2200640 1 --2- 172.21.15.49:0/4134069649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 0x7f89bc06e8c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0b5aa0 tx=0x7f89b002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.074+0000 7f89c2200640 1 -- 172.21.15.49:0/4134069649 shutdown_connections 2024-04-15T15:18:46.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.074+0000 7f89c2200640 1 --2- 172.21.15.49:0/4134069649 >> v2:172.21.15.107:3300/0 conn(0x7f89bc071f50 0x7f89bc0723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.074+0000 7f89c2200640 1 --2- 172.21.15.49:0/4134069649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 0x7f89bc06e8c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.074+0000 7f89c2200640 1 -- 172.21.15.49:0/4134069649 >> 172.21.15.49:0/4134069649 conn(0x7f89bc06b930 msgr2=0x7f89bc06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:46.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.074+0000 7f89c2200640 1 -- 172.21.15.49:0/4134069649 shutdown_connections 2024-04-15T15:18:46.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.074+0000 7f89c2200640 1 -- 172.21.15.49:0/4134069649 wait complete. 2024-04-15T15:18:46.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.076+0000 7f89c2200640 1 Processor -- start 2024-04-15T15:18:46.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.076+0000 7f89c2200640 1 -- start start 2024-04-15T15:18:46.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.077+0000 7f89c2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89bc071f50 0x7f89bc153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.077+0000 7f89c2200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 0x7f89bc154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.077+0000 7f89c2200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89bc15cbe0 0x7f89bc14cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.077+0000 7f89c2200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc06e3a0 con 0x7f89bc0cee00 2024-04-15T15:18:46.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.077+0000 7f89c2200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc06e0c0 con 0x7f89bc071f50 2024-04-15T15:18:46.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.077+0000 7f89c2200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f89bc15dda0 con 0x7f89bc15cbe0 2024-04-15T15:18:46.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.077+0000 7f89bbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 0x7f89bc154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:46.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.077+0000 7f89bbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 0x7f89bc154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58212/0 (socket says 172.21.15.49:58212) 2024-04-15T15:18:46.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.077+0000 7f89bbe00640 1 -- 172.21.15.49:0/42236215 learned_addr learned my addr 172.21.15.49:0/42236215 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89c0e00640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89bc071f50 0x7f89bc153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89bbe00640 1 -- 172.21.15.49:0/42236215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89bc15cbe0 msgr2=0x7f89bc14cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89bbe00640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89bc15cbe0 0x7f89bc14cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89bbe00640 1 -- 172.21.15.49:0/42236215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89bc071f50 msgr2=0x7f89bc153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89bbe00640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89bc071f50 0x7f89bc153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89bbe00640 1 -- 172.21.15.49:0/42236215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc1523b0 con 0x7f89bc0cee00 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89c0e00640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89bc071f50 0x7f89bc153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89bbe00640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 0x7f89bc154310 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f89a800e9b0 tx=0x7f89a800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89b8c00640 1 -- 172.21.15.49:0/42236215 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8017070 con 0x7f89bc0cee00 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89b8c00640 1 -- 172.21.15.49:0/42236215 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a800c9a0 con 0x7f89bc0cee00 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89b8c00640 1 -- 172.21.15.49:0/42236215 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a800cd90 con 0x7f89bc0cee00 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.078+0000 7f89c2200640 1 -- 172.21.15.49:0/42236215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89bc152690 con 0x7f89bc0cee00 2024-04-15T15:18:46.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.079+0000 7f89c2200640 1 -- 172.21.15.49:0/42236215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89bc152c50 con 0x7f89bc0cee00 2024-04-15T15:18:46.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.080+0000 7f89c2200640 1 -- 172.21.15.49:0/42236215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89bc0755c0 con 0x7f89bc0cee00 2024-04-15T15:18:46.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.081+0000 7f89b8c00640 1 -- 172.21.15.49:0/42236215 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89a801a070 con 0x7f89bc0cee00 2024-04-15T15:18:46.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.082+0000 7f89b8c00640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f899c06c8f0 0x7f899c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.082+0000 7f89b8c00640 1 -- 172.21.15.49:0/42236215 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f89a808bef0 con 0x7f89bc0cee00 2024-04-15T15:18:46.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.082+0000 7f89c0e00640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f899c06c8f0 0x7f899c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:46.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.083+0000 7f89c0e00640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f899c06c8f0 0x7f899c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f89b0036000 tx=0x7f89b0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:46.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.086+0000 7f89b8c00640 1 -- 172.21.15.49:0/42236215 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89a805a310 con 0x7f89bc0cee00 2024-04-15T15:18:46.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.291+0000 7f89c2200640 1 -- 172.21.15.49:0/42236215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89bc06fb00 con 0x7f89bc0cee00 2024-04-15T15:18:46.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.292+0000 7f89b8c00640 1 -- 172.21.15.49:0/42236215 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16618 (secure 0 0 0) 0x7f89a8059c10 con 0x7f89bc0cee00 2024-04-15T15:18:46.294 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:46.295 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:45.326076+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 -- 172.21.15.49:0/42236215 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f899c06c8f0 msgr2=0x7f899c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f899c06c8f0 0x7f899c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f89b0036000 tx=0x7f89b0008030 comp rx=0 tx=0).stop 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 -- 172.21.15.49:0/42236215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 msgr2=0x7f89bc154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 0x7f89bc154310 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f89a800e9b0 tx=0x7f89a800ee70 comp rx=0 tx=0).stop 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 -- 172.21.15.49:0/42236215 shutdown_connections 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f899c06c8f0 0x7f899c06edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89bc15cbe0 0x7f89bc14cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89bc0cee00 0x7f89bc154310 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 --2- 172.21.15.49:0/42236215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89bc071f50 0x7f89bc153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 -- 172.21.15.49:0/42236215 >> 172.21.15.49:0/42236215 conn(0x7f89bc06b930 msgr2=0x7f89bc159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:46.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 -- 172.21.15.49:0/42236215 shutdown_connections 2024-04-15T15:18:46.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.295+0000 7f89c2200640 1 -- 172.21.15.49:0/42236215 wait complete. 2024-04-15T15:18:46.315 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:46.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.580+0000 7f30c9000640 1 Processor -- start 2024-04-15T15:18:46.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.580+0000 7f30c9000640 1 -- start start 2024-04-15T15:18:46.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.582+0000 7f30c9000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30c40affc0 0x7f30c415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.582+0000 7f30c9000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c41485d0 con 0x7f30c41481f0 2024-04-15T15:18:46.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.582+0000 7f30c9000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c40aef20 con 0x7f30c40728f0 2024-04-15T15:18:46.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.582+0000 7f30c9000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f30c415ffc0 con 0x7f30c40affc0 2024-04-15T15:18:46.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.582+0000 7f30c1e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f30c41481f0 0x7f30c40721e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43998/0 (socket says 172.21.15.49:43998) 2024-04-15T15:18:46.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.582+0000 7f30c1e00640 1 -- 172.21.15.49:0/3242333329 learned_addr learned my addr 172.21.15.49:0/3242333329 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:46.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.582+0000 7f30c2800640 1 --2- 172.21.15.49:0/3242333329 >> v2:172.21.15.49:3300/0 conn(0x7f30c40affc0 0x7f30c415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.583+0000 7f30c0a00640 1 -- 172.21.15.49:0/3242333329 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 622172231 0 0) 0x7f30c41485d0 con 0x7f30c41481f0 2024-04-15T15:18:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.583+0000 7f30c0a00640 1 -- 172.21.15.49:0/3242333329 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30a8003520 con 0x7f30c41481f0 2024-04-15T15:18:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.583+0000 7f30c2800640 1 -- 172.21.15.49:0/3242333329 >> v1:172.21.15.107:6789/0 conn(0x7f30c40728f0 legacy=0x7f30c41601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.583+0000 7f30c2800640 1 -- 172.21.15.49:0/3242333329 >> v1:172.21.15.49:6789/0 conn(0x7f30c41481f0 legacy=0x7f30c40721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.583+0000 7f30c2800640 1 -- 172.21.15.49:0/3242333329 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c40af200 con 0x7f30c40affc0 2024-04-15T15:18:46.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.584+0000 7f30c2800640 1 --2- 172.21.15.49:0/3242333329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40affc0 0x7f30c415fa80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009dc0 tx=0x7f30bc02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc16efc49b5fec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:46.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.585+0000 7f30c0a00640 1 -- 172.21.15.49:0/3242333329 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc039070 con 0x7f30c40affc0 2024-04-15T15:18:46.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.585+0000 7f30c0a00640 1 -- 172.21.15.49:0/3242333329 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30bc02cc40 con 0x7f30c40affc0 2024-04-15T15:18:46.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.585+0000 7f30c0a00640 1 -- 172.21.15.49:0/3242333329 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc034440 con 0x7f30c40affc0 2024-04-15T15:18:46.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.585+0000 7f30c9000640 1 -- 172.21.15.49:0/3242333329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40affc0 msgr2=0x7f30c415fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.585+0000 7f30c9000640 1 --2- 172.21.15.49:0/3242333329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40affc0 0x7f30c415fa80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009dc0 tx=0x7f30bc02be90 comp rx=0 tx=0).stop 2024-04-15T15:18:46.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.585+0000 7f30c9000640 1 -- 172.21.15.49:0/3242333329 shutdown_connections 2024-04-15T15:18:46.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.585+0000 7f30c9000640 1 --2- 172.21.15.49:0/3242333329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40affc0 0x7f30c415fa80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.585+0000 7f30c9000640 1 -- 172.21.15.49:0/3242333329 >> 172.21.15.49:0/3242333329 conn(0x7f30c40b27b0 msgr2=0x7f30c40b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:46.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.586+0000 7f30c9000640 1 -- 172.21.15.49:0/3242333329 shutdown_connections 2024-04-15T15:18:46.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.586+0000 7f30c9000640 1 -- 172.21.15.49:0/3242333329 wait complete. 2024-04-15T15:18:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.587+0000 7f30c9000640 1 Processor -- start 2024-04-15T15:18:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.587+0000 7f30c9000640 1 -- start start 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.588+0000 7f30c9000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40728f0 0x7f30c415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.588+0000 7f30c9000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30c40affc0 0x7f30c406e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.588+0000 7f30c9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30c41481f0 0x7f30c4145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.588+0000 7f30c9000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30c40afaa0 con 0x7f30c40728f0 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.588+0000 7f30c9000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f30c40af7c0 con 0x7f30c41481f0 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.588+0000 7f30c9000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f30c40b5560 con 0x7f30c40affc0 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.589+0000 7f30c1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40728f0 0x7f30c415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.589+0000 7f30c1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40728f0 0x7f30c415e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58230/0 (socket says 172.21.15.49:58230) 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.589+0000 7f30c1e00640 1 -- 172.21.15.49:0/357956425 learned_addr learned my addr 172.21.15.49:0/357956425 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.589+0000 7f30c1e00640 1 -- 172.21.15.49:0/357956425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30c40affc0 msgr2=0x7f30c406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.589+0000 7f30c1e00640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30c40affc0 0x7f30c406e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.589+0000 7f30c1e00640 1 -- 172.21.15.49:0/357956425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30c41481f0 msgr2=0x7f30c4145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.589+0000 7f30c1e00640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30c41481f0 0x7f30c4145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.589+0000 7f30c1e00640 1 -- 172.21.15.49:0/357956425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c406e780 con 0x7f30c40728f0 2024-04-15T15:18:46.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.589+0000 7f30c1e00640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40728f0 0x7f30c415e250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f30b400c410 tx=0x7f30b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:46.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.589+0000 7f30ba000640 1 -- 172.21.15.49:0/357956425 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4017070 con 0x7f30c40728f0 2024-04-15T15:18:46.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.590+0000 7f30ba000640 1 -- 172.21.15.49:0/357956425 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b4005ce0 con 0x7f30c40728f0 2024-04-15T15:18:46.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.590+0000 7f30ba000640 1 -- 172.21.15.49:0/357956425 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b40072f0 con 0x7f30c40728f0 2024-04-15T15:18:46.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.590+0000 7f30c9000640 1 -- 172.21.15.49:0/357956425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c406ea20 con 0x7f30c40728f0 2024-04-15T15:18:46.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.590+0000 7f30c9000640 1 -- 172.21.15.49:0/357956425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c406efe0 con 0x7f30c40728f0 2024-04-15T15:18:46.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.591+0000 7f30c9000640 1 -- 172.21.15.49:0/357956425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c4068c80 con 0x7f30c40728f0 2024-04-15T15:18:46.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.592+0000 7f30ba000640 1 -- 172.21.15.49:0/357956425 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30b401a070 con 0x7f30c40728f0 2024-04-15T15:18:46.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.592+0000 7f30ba000640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30a406c8f0 0x7f30a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:46.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.593+0000 7f30ba000640 1 -- 172.21.15.49:0/357956425 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f30b408b080 con 0x7f30c40728f0 2024-04-15T15:18:46.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.593+0000 7f30c1400640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30a406c8f0 0x7f30a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:46.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.593+0000 7f30c1400640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30a406c8f0 0x7f30a406edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f30b0000f60 tx=0x7f30b00079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:46.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.597+0000 7f30ba000640 1 -- 172.21.15.49:0/357956425 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30b40594d0 con 0x7f30c40728f0 2024-04-15T15:18:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.803+0000 7f30c9000640 1 -- 172.21.15.49:0/357956425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30c4145280 con 0x7f30c40728f0 2024-04-15T15:18:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.806+0000 7f30ba000640 1 -- 172.21.15.49:0/357956425 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16618 (secure 0 0 0) 0x7f30b4058dd0 con 0x7f30c40728f0 2024-04-15T15:18:46.809 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:46.809 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:45.326076+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.809+0000 7f30c9000640 1 -- 172.21.15.49:0/357956425 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30a406c8f0 msgr2=0x7f30a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.809+0000 7f30c9000640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30a406c8f0 0x7f30a406edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f30b0000f60 tx=0x7f30b00079d0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.809+0000 7f30c9000640 1 -- 172.21.15.49:0/357956425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40728f0 msgr2=0x7f30c415e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.809+0000 7f30c9000640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40728f0 0x7f30c415e250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f30b400c410 tx=0x7f30b400c8d0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.810+0000 7f30c9000640 1 -- 172.21.15.49:0/357956425 shutdown_connections 2024-04-15T15:18:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.810+0000 7f30c9000640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30a406c8f0 0x7f30a406edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.810+0000 7f30c9000640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30c40affc0 0x7f30c406e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.810+0000 7f30c9000640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30c41481f0 0x7f30c4145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.810+0000 7f30c9000640 1 --2- 172.21.15.49:0/357956425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30c40728f0 0x7f30c415e250 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.810+0000 7f30c9000640 1 -- 172.21.15.49:0/357956425 >> 172.21.15.49:0/357956425 conn(0x7f30c40b27b0 msgr2=0x7f30c40b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.810+0000 7f30c9000640 1 -- 172.21.15.49:0/357956425 shutdown_connections 2024-04-15T15:18:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:46.810+0000 7f30c9000640 1 -- 172.21.15.49:0/357956425 wait complete. 2024-04-15T15:18:46.829 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-15T15:18:47.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.049+0000 7fdc5f200640 1 Processor -- start 2024-04-15T15:18:47.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.049+0000 7fdc5f200640 1 -- start start 2024-04-15T15:18:47.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.051+0000 7fdc5f200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdc5815e880 0x7fdc58162e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:47.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.051+0000 7fdc5f200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc5806e990 con 0x7fdc5806f740 2024-04-15T15:18:47.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.051+0000 7fdc5f200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc5815ec60 con 0x7fdc5806ec10 2024-04-15T15:18:47.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.051+0000 7fdc5f200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fdc580add80 con 0x7fdc5815e880 2024-04-15T15:18:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.051+0000 7fdc5e800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdc5806f740 0x7fdc5806c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44014/0 (socket says 172.21.15.49:44014) 2024-04-15T15:18:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.051+0000 7fdc5e800640 1 -- 172.21.15.49:0/64130237 learned_addr learned my addr 172.21.15.49:0/64130237 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5de00640 1 --2- 172.21.15.49:0/64130237 >> v2:172.21.15.107:3300/0 conn(0x7fdc5815e880 0x7fdc58162e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1142559611 0 0) 0x7fdc5806e990 con 0x7fdc5806f740 2024-04-15T15:18:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc40003520 con 0x7fdc5806f740 2024-04-15T15:18:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2046131900 0 0) 0x7fdc40003520 con 0x7fdc5806f740 2024-04-15T15:18:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 >> v2:172.21.15.107:3300/0 conn(0x7fdc5815e880 msgr2=0x7fdc58162e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 --2- 172.21.15.49:0/64130237 >> v2:172.21.15.107:3300/0 conn(0x7fdc5815e880 0x7fdc58162e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 >> v1:172.21.15.107:6789/0 conn(0x7fdc5806ec10 legacy=0x7fdc5806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc580b0340 con 0x7fdc5806f740 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fdc540019f0 con 0x7fdc5806f740 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 >> v1:172.21.15.49:6789/0 conn(0x7fdc5806f740 legacy=0x7fdc5806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 --2- 172.21.15.49:0/64130237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc40003a00 0x7fdc40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 --2- 172.21.15.49:0/64130237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc40004800 0x7fdc40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 --2- 172.21.15.49:0/64130237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc400051c0 0x7fdc4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4000c090 con 0x7fdc400051c0 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4000c240 con 0x7fdc40004800 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.052+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4000c440 con 0x7fdc40003a00 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.053+0000 7fdc5e800640 1 --2- 172.21.15.49:0/64130237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc400051c0 0x7fdc4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.053+0000 7fdc5de00640 1 --2- 172.21.15.49:0/64130237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc40003a00 0x7fdc40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.053+0000 7fdc5e800640 1 -- 172.21.15.49:0/64130237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc40003a00 msgr2=0x7fdc40003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.053+0000 7fdc5e800640 1 --2- 172.21.15.49:0/64130237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc40003a00 0x7fdc40003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.053+0000 7fdc5e800640 1 -- 172.21.15.49:0/64130237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc40004800 msgr2=0x7fdc40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:47.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.053+0000 7fdc5e800640 1 --2- 172.21.15.49:0/64130237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc40004800 0x7fdc40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.053+0000 7fdc5e800640 1 -- 172.21.15.49:0/64130237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc40003520 con 0x7fdc400051c0 2024-04-15T15:18:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.054+0000 7fdc5e800640 1 -- 172.21.15.49:0/64130237 reap_dead start 2024-04-15T15:18:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.054+0000 7fdc5e800640 1 --2- 172.21.15.49:0/64130237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc400051c0 0x7fdc4000b9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400a7b0 tx=0x7fdc40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f3a3b24f35162 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.054+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400a3d0 con 0x7fdc400051c0 2024-04-15T15:18:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.054+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc54034650 con 0x7fdc400051c0 2024-04-15T15:18:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.054+0000 7fdc5ca00640 1 -- 172.21.15.49:0/64130237 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54034a40 con 0x7fdc400051c0 2024-04-15T15:18:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.055+0000 7fdc5f200640 1 -- 172.21.15.49:0/64130237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc400051c0 msgr2=0x7fdc4000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.055+0000 7fdc5f200640 1 --2- 172.21.15.49:0/64130237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc400051c0 0x7fdc4000b9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400a7b0 tx=0x7fdc40005830 comp rx=0 tx=0).stop 2024-04-15T15:18:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.055+0000 7fdc5f200640 1 -- 172.21.15.49:0/64130237 shutdown_connections 2024-04-15T15:18:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.055+0000 7fdc5f200640 1 --2- 172.21.15.49:0/64130237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc400051c0 0x7fdc4000b9c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.055+0000 7fdc5f200640 1 -- 172.21.15.49:0/64130237 >> 172.21.15.49:0/64130237 conn(0x7fdc580b29a0 msgr2=0x7fdc580b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.055+0000 7fdc5f200640 1 -- 172.21.15.49:0/64130237 shutdown_connections 2024-04-15T15:18:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.055+0000 7fdc5f200640 1 -- 172.21.15.49:0/64130237 wait complete. 2024-04-15T15:18:47.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.057+0000 7fdc5f200640 1 Processor -- start 2024-04-15T15:18:47.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.057+0000 7fdc5f200640 1 -- start start 2024-04-15T15:18:47.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.058+0000 7fdc5f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc400051c0 0x7fdc58154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:47.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.058+0000 7fdc5f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc5815ee90 0x7fdc58152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:47.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.058+0000 7fdc5f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc581f7ce0 0x7fdc581f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.058+0000 7fdc5f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5806bc10 con 0x7fdc5815ee90 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.058+0000 7fdc5f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5806b930 con 0x7fdc581f7ce0 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.058+0000 7fdc5f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdc58162e20 con 0x7fdc400051c0 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.058+0000 7fdc5d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc5815ee90 0x7fdc58152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.058+0000 7fdc5d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc5815ee90 0x7fdc58152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58258/0 (socket says 172.21.15.49:58258) 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.058+0000 7fdc5d400640 1 -- 172.21.15.49:0/1792666998 learned_addr learned my addr 172.21.15.49:0/1792666998 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc5de00640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc400051c0 0x7fdc58154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc5d400640 1 -- 172.21.15.49:0/1792666998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc400051c0 msgr2=0x7fdc58154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc5d400640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc400051c0 0x7fdc58154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc5d400640 1 -- 172.21.15.49:0/1792666998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc581f7ce0 msgr2=0x7fdc581f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc5d400640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc581f7ce0 0x7fdc581f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc5d400640 1 -- 172.21.15.49:0/1792666998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc581525a0 con 0x7fdc5815ee90 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc5d400640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc5815ee90 0x7fdc58152060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800c500 tx=0x7fdc4800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc52000640 1 -- 172.21.15.49:0/1792666998 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc480052d0 con 0x7fdc5815ee90 2024-04-15T15:18:47.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc52000640 1 -- 172.21.15.49:0/1792666998 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc48005470 con 0x7fdc5815ee90 2024-04-15T15:18:47.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc52000640 1 -- 172.21.15.49:0/1792666998 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48012870 con 0x7fdc5815ee90 2024-04-15T15:18:47.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc5f200640 1 -- 172.21.15.49:0/1792666998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc58155c60 con 0x7fdc5815ee90 2024-04-15T15:18:47.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.059+0000 7fdc5f200640 1 -- 172.21.15.49:0/1792666998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc58155980 con 0x7fdc5815ee90 2024-04-15T15:18:47.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.060+0000 7fdc5f200640 1 -- 172.21.15.49:0/1792666998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc580755c0 con 0x7fdc5815ee90 2024-04-15T15:18:47.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.061+0000 7fdc52000640 1 -- 172.21.15.49:0/1792666998 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc48012a10 con 0x7fdc5815ee90 2024-04-15T15:18:47.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.062+0000 7fdc52000640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:47.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.062+0000 7fdc52000640 1 -- 172.21.15.49:0/1792666998 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdc4808b790 con 0x7fdc5815ee90 2024-04-15T15:18:47.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.063+0000 7fdc5de00640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:47.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.063+0000 7fdc5de00640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0098d0 tx=0x7fdc4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:47.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.067+0000 7fdc52000640 1 -- 172.21.15.49:0/1792666998 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc48059c50 con 0x7fdc5815ee90 2024-04-15T15:18:47.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.315+0000 7fdc5f200640 1 -- 172.21.15.49:0/1792666998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdc581631b0 con 0x7fdc5815ee90 2024-04-15T15:18:47.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.859+0000 7fdc52000640 1 -- 172.21.15.49:0/1792666998 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fdc48059550 con 0x7fdc5815ee90 2024-04-15T15:18:47.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.866+0000 7fdc5f200640 1 -- 172.21.15.49:0/1792666998 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdc3c06c8f0 msgr2=0x7fdc3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:47.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.866+0000 7fdc5f200640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0098d0 tx=0x7fdc4c008040 comp rx=0 tx=0).stop 2024-04-15T15:18:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.866+0000 7fdc5f200640 1 -- 172.21.15.49:0/1792666998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc5815ee90 msgr2=0x7fdc58152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.866+0000 7fdc5f200640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc5815ee90 0x7fdc58152060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800c500 tx=0x7fdc4800c9c0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.867+0000 7fdc5f200640 1 -- 172.21.15.49:0/1792666998 shutdown_connections 2024-04-15T15:18:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.867+0000 7fdc5f200640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.867+0000 7fdc5f200640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc581f7ce0 0x7fdc581f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.867+0000 7fdc5f200640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc5815ee90 0x7fdc58152060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.867+0000 7fdc5f200640 1 --2- 172.21.15.49:0/1792666998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc400051c0 0x7fdc58154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:47.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.867+0000 7fdc5f200640 1 -- 172.21.15.49:0/1792666998 >> 172.21.15.49:0/1792666998 conn(0x7fdc580b29a0 msgr2=0x7fdc580ade30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:47.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.868+0000 7fdc5f200640 1 -- 172.21.15.49:0/1792666998 shutdown_connections 2024-04-15T15:18:47.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:47.868+0000 7fdc5f200640 1 -- 172.21.15.49:0/1792666998 wait complete. 2024-04-15T15:18:47.891 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-15T15:18:47.891 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-15T15:18:48.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.109+0000 7f2f93e00640 1 Processor -- start 2024-04-15T15:18:48.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.109+0000 7f2f93e00640 1 -- start start 2024-04-15T15:18:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.111+0000 7f2f93e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2f941478b0 0x7f2f940afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.111+0000 7f2f93e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2f9406f920 0x7f2f941626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.111+0000 7f2f93e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f940aebd0 con 0x7f2f94147c90 2024-04-15T15:18:48.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.111+0000 7f2f93e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f2f940b0130 con 0x7f2f941478b0 2024-04-15T15:18:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.111+0000 7f2f93e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f2f940aee50 con 0x7f2f9406f920 2024-04-15T15:18:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.111+0000 7f2f92a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2f941478b0 0x7f2f940afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.111+0000 7f2f8be00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2f94147c90 0x7f2f9406f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44030/0 (socket says 172.21.15.49:44030) 2024-04-15T15:18:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.111+0000 7f2f8be00640 1 -- 172.21.15.49:0/4139052372 learned_addr learned my addr 172.21.15.49:0/4139052372 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.112+0000 7f2f93400640 1 --2- 172.21.15.49:0/4139052372 >> v2:172.21.15.107:3300/0 conn(0x7f2f9406f920 0x7f2f941626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.112+0000 7f2f92a00640 1 -- 172.21.15.49:0/4139052372 >> v2:172.21.15.107:3300/0 conn(0x7f2f9406f920 msgr2=0x7f2f941626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.112+0000 7f2f92a00640 1 --2- 172.21.15.49:0/4139052372 >> v2:172.21.15.107:3300/0 conn(0x7f2f9406f920 0x7f2f941626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.112+0000 7f2f92000640 1 -- 172.21.15.49:0/4139052372 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2439427912 0 0) 0x7f2f7c001110 con 0x7f2f94147c90 2024-04-15T15:18:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.112+0000 7f2f92a00640 1 -- 172.21.15.49:0/4139052372 >> v1:172.21.15.49:6789/0 conn(0x7f2f94147c90 legacy=0x7f2f9406f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:48.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.112+0000 7f2f92a00640 1 -- 172.21.15.49:0/4139052372 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f940af110 con 0x7f2f941478b0 2024-04-15T15:18:48.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.114+0000 7f2f92a00640 1 --2- 172.21.15.49:0/4139052372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f941478b0 0x7f2f940afbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f9406c860 tx=0x7f2f8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcdd9fda00ebe11 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:48.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.114+0000 7f2f92000640 1 -- 172.21.15.49:0/4139052372 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84035020 con 0x7f2f941478b0 2024-04-15T15:18:48.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.114+0000 7f2f92000640 1 -- 172.21.15.49:0/4139052372 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f84031830 con 0x7f2f941478b0 2024-04-15T15:18:48.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.115+0000 7f2f92000640 1 -- 172.21.15.49:0/4139052372 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84031c00 con 0x7f2f941478b0 2024-04-15T15:18:48.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.115+0000 7f2f93e00640 1 -- 172.21.15.49:0/4139052372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f941478b0 msgr2=0x7f2f940afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:48.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.115+0000 7f2f93e00640 1 --2- 172.21.15.49:0/4139052372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f941478b0 0x7f2f940afbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f9406c860 tx=0x7f2f8402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.115+0000 7f2f93e00640 1 -- 172.21.15.49:0/4139052372 shutdown_connections 2024-04-15T15:18:48.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.115+0000 7f2f93e00640 1 --2- 172.21.15.49:0/4139052372 >> v2:172.21.15.107:3300/0 conn(0x7f2f9406f920 0x7f2f941626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.115+0000 7f2f93e00640 1 --2- 172.21.15.49:0/4139052372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f941478b0 0x7f2f940afbf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.115+0000 7f2f93e00640 1 -- 172.21.15.49:0/4139052372 >> 172.21.15.49:0/4139052372 conn(0x7f2f9406b930 msgr2=0x7f2f9406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:48.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.115+0000 7f2f93e00640 1 -- 172.21.15.49:0/4139052372 shutdown_connections 2024-04-15T15:18:48.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.115+0000 7f2f93e00640 1 -- 172.21.15.49:0/4139052372 wait complete. 2024-04-15T15:18:48.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.117+0000 7f2f93e00640 1 Processor -- start 2024-04-15T15:18:48.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.117+0000 7f2f93e00640 1 -- start start 2024-04-15T15:18:48.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.118+0000 7f2f93e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f9406f920 0x7f2f9415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:48.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.118+0000 7f2f93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f941478b0 0x7f2f940cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.118+0000 7f2f93e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f94147c90 0x7f2f941464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.118+0000 7f2f93e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2f94162c00 con 0x7f2f9406f920 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.118+0000 7f2f93e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f940af6d0 con 0x7f2f941478b0 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.118+0000 7f2f93e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f2f94162d80 con 0x7f2f94147c90 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.118+0000 7f2f92a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f9406f920 0x7f2f9415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.118+0000 7f2f92a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f9406f920 0x7f2f9415e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58272/0 (socket says 172.21.15.49:58272) 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.118+0000 7f2f92a00640 1 -- 172.21.15.49:0/1703162189 learned_addr learned my addr 172.21.15.49:0/1703162189 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.119+0000 7f2f8be00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f941478b0 0x7f2f940cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.119+0000 7f2f93400640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f94147c90 0x7f2f941464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.119+0000 7f2f92a00640 1 -- 172.21.15.49:0/1703162189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f94147c90 msgr2=0x7f2f941464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.119+0000 7f2f92a00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f94147c90 0x7f2f941464f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.119+0000 7f2f92a00640 1 -- 172.21.15.49:0/1703162189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f941478b0 msgr2=0x7f2f940cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:48.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.119+0000 7f2f92a00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f941478b0 0x7f2f940cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.119+0000 7f2f92a00640 1 -- 172.21.15.49:0/1703162189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9415e740 con 0x7f2f9406f920 2024-04-15T15:18:48.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.119+0000 7f2f8be00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f941478b0 0x7f2f940cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:48.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.119+0000 7f2f92a00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f9406f920 0x7f2f9415e200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f84000c00 tx=0x7f2f840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:48.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.119+0000 7f2f8b400640 1 -- 172.21.15.49:0/1703162189 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84035070 con 0x7f2f9406f920 2024-04-15T15:18:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.120+0000 7f2f8b400640 1 -- 172.21.15.49:0/1703162189 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f84004250 con 0x7f2f9406f920 2024-04-15T15:18:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.120+0000 7f2f93e00640 1 -- 172.21.15.49:0/1703162189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f940cfa00 con 0x7f2f9406f920 2024-04-15T15:18:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.120+0000 7f2f8b400640 1 -- 172.21.15.49:0/1703162189 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8403a6f0 con 0x7f2f9406f920 2024-04-15T15:18:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.120+0000 7f2f93e00640 1 -- 172.21.15.49:0/1703162189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f940ce700 con 0x7f2f9406f920 2024-04-15T15:18:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.121+0000 7f2f93e00640 1 -- 172.21.15.49:0/1703162189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9406c8c0 con 0x7f2f9406f920 2024-04-15T15:18:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.122+0000 7f2f8b400640 1 -- 172.21.15.49:0/1703162189 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f84038020 con 0x7f2f9406f920 2024-04-15T15:18:48.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.123+0000 7f2f8b400640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2f7006c8f0 0x7f2f7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:48.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.123+0000 7f2f8b400640 1 -- 172.21.15.49:0/1703162189 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2f840ad830 con 0x7f2f9406f920 2024-04-15T15:18:48.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.123+0000 7f2f8be00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2f7006c8f0 0x7f2f7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:48.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.124+0000 7f2f8be00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2f7006c8f0 0x7f2f7006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2f941f7b00 tx=0x7f2f7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:48.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.127+0000 7f2f8b400640 1 -- 172.21.15.49:0/1703162189 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f9415e740 con 0x7f2f9406f920 2024-04-15T15:18:48.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.376+0000 7f2f93e00640 1 -- 172.21.15.49:0/1703162189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2f940755c0 con 0x7f2f9406f920 2024-04-15T15:18:48.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.862+0000 7f2f8b400640 1 -- 172.21.15.49:0/1703162189 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f2f84040070 con 0x7f2f9406f920 2024-04-15T15:18:48.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.866+0000 7f2f93e00640 1 -- 172.21.15.49:0/1703162189 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2f7006c8f0 msgr2=0x7f2f7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:48.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.866+0000 7f2f93e00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2f7006c8f0 0x7f2f7006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2f941f7b00 tx=0x7f2f7c009040 comp rx=0 tx=0).stop 2024-04-15T15:18:48.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.866+0000 7f2f93e00640 1 -- 172.21.15.49:0/1703162189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f9406f920 msgr2=0x7f2f9415e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:48.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.866+0000 7f2f93e00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f9406f920 0x7f2f9415e200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f84000c00 tx=0x7f2f840046c0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.866+0000 7f2f93e00640 1 -- 172.21.15.49:0/1703162189 shutdown_connections 2024-04-15T15:18:48.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.866+0000 7f2f93e00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2f7006c8f0 0x7f2f7006edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.866+0000 7f2f93e00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f94147c90 0x7f2f941464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.866+0000 7f2f93e00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f941478b0 0x7f2f940cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.866+0000 7f2f93e00640 1 --2- 172.21.15.49:0/1703162189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f9406f920 0x7f2f9415e200 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:48.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.866+0000 7f2f93e00640 1 -- 172.21.15.49:0/1703162189 >> 172.21.15.49:0/1703162189 conn(0x7f2f9406b930 msgr2=0x7f2f940cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:48.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.867+0000 7f2f93e00640 1 -- 172.21.15.49:0/1703162189 shutdown_connections 2024-04-15T15:18:48.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:48.867+0000 7f2f93e00640 1 -- 172.21.15.49:0/1703162189 wait complete. 2024-04-15T15:18:48.894 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-15T15:18:48.894 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:18:48.962 INFO:tasks.ceph.mds.d.smithi107.stderr:2024-04-15T15:18:48.961+0000 7ffa10c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:18:49.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.124+0000 7f30ffe00640 1 Processor -- start 2024-04-15T15:18:49.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.124+0000 7f30ffe00640 1 -- start start 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.125+0000 7f30ffe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3100072660 0x7f310006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.125+0000 7f30ffe00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f310015cbe0 0x7f310015d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30ffe00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f310015dbb0 con 0x7f31000cee00 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30ffe00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f310006f4f0 con 0x7f3100072660 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30ffe00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f310015d8d0 con 0x7f310015cbe0 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30fe000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3100072660 0x7f310006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30fe000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3100072660 0x7f310006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58284/0 (socket says 172.21.15.49:58284) 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30fe000640 1 -- 172.21.15.49:0/3571684350 learned_addr learned my addr 172.21.15.49:0/3571684350 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30ff400640 1 --2- 172.21.15.49:0/3571684350 >> v2:172.21.15.107:3300/0 conn(0x7f310015cbe0 0x7f310015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30fe000640 1 -- 172.21.15.49:0/3571684350 >> v2:172.21.15.107:3300/0 conn(0x7f310015cbe0 msgr2=0x7f310015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30fe000640 1 --2- 172.21.15.49:0/3571684350 >> v2:172.21.15.107:3300/0 conn(0x7f310015cbe0 0x7f310015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30fe000640 1 -- 172.21.15.49:0/3571684350 >> v1:172.21.15.107:6789/0 conn(0x7f31000cee00 legacy=0x7f3100071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.126+0000 7f30fe000640 1 -- 172.21.15.49:0/3571684350 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310006dde0 con 0x7f3100072660 2024-04-15T15:18:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.128+0000 7f30fe000640 1 --2- 172.21.15.49:0/3571684350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 0x7f310006e8c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31000b02a0 tx=0x7f30ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a76f70566cd91aa server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.128+0000 7f30fd600640 1 -- 172.21.15.49:0/3571684350 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec035020 con 0x7f3100072660 2024-04-15T15:18:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.128+0000 7f30fd600640 1 -- 172.21.15.49:0/3571684350 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ec031870 con 0x7f3100072660 2024-04-15T15:18:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.128+0000 7f30fd600640 1 -- 172.21.15.49:0/3571684350 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec031c40 con 0x7f3100072660 2024-04-15T15:18:49.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.128+0000 7f30ffe00640 1 -- 172.21.15.49:0/3571684350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 msgr2=0x7f310006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:49.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.128+0000 7f30ffe00640 1 --2- 172.21.15.49:0/3571684350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 0x7f310006e8c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31000b02a0 tx=0x7f30ec02b910 comp rx=0 tx=0).stop 2024-04-15T15:18:49.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.129+0000 7f30ffe00640 1 -- 172.21.15.49:0/3571684350 shutdown_connections 2024-04-15T15:18:49.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.129+0000 7f30ffe00640 1 --2- 172.21.15.49:0/3571684350 >> v2:172.21.15.107:3300/0 conn(0x7f310015cbe0 0x7f310015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.129+0000 7f30ffe00640 1 --2- 172.21.15.49:0/3571684350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 0x7f310006e8c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.129+0000 7f30ffe00640 1 -- 172.21.15.49:0/3571684350 >> 172.21.15.49:0/3571684350 conn(0x7f310006b930 msgr2=0x7f310006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:49.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.129+0000 7f30ffe00640 1 -- 172.21.15.49:0/3571684350 shutdown_connections 2024-04-15T15:18:49.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.129+0000 7f30ffe00640 1 -- 172.21.15.49:0/3571684350 wait complete. 2024-04-15T15:18:49.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.130+0000 7f30ffe00640 1 Processor -- start 2024-04-15T15:18:49.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.131+0000 7f30ffe00640 1 -- start start 2024-04-15T15:18:49.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.131+0000 7f30ffe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 0x7f310014e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31000cee00 0x7f310015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30ffe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f310015cbe0 0x7f310015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30ffe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f310015d5b0 con 0x7f3100072660 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30ffe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f310006e3a0 con 0x7f31000cee00 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30ffe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f310015dda0 con 0x7f310015cbe0 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30fea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 0x7f310014e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30fea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 0x7f310014e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58294/0 (socket says 172.21.15.49:58294) 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30fea00640 1 -- 172.21.15.49:0/570405806 learned_addr learned my addr 172.21.15.49:0/570405806 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30fe000640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31000cee00 0x7f310015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30fea00640 1 -- 172.21.15.49:0/570405806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f310015cbe0 msgr2=0x7f310015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30fea00640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f310015cbe0 0x7f310015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30fea00640 1 -- 172.21.15.49:0/570405806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31000cee00 msgr2=0x7f310015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30fea00640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31000cee00 0x7f310015f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30fea00640 1 -- 172.21.15.49:0/570405806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310014ebf0 con 0x7f3100072660 2024-04-15T15:18:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.132+0000 7f30fe000640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31000cee00 0x7f310015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.133+0000 7f30fea00640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 0x7f310014e6b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f30f000c7c0 tx=0x7f30f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.133+0000 7f30f6a00640 1 -- 172.21.15.49:0/570405806 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0016020 con 0x7f3100072660 2024-04-15T15:18:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.133+0000 7f30f6a00640 1 -- 172.21.15.49:0/570405806 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f000aa50 con 0x7f3100072660 2024-04-15T15:18:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.133+0000 7f30f6a00640 1 -- 172.21.15.49:0/570405806 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0012420 con 0x7f3100072660 2024-04-15T15:18:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.133+0000 7f30ffe00640 1 -- 172.21.15.49:0/570405806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31001580b0 con 0x7f3100072660 2024-04-15T15:18:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.133+0000 7f30ffe00640 1 -- 172.21.15.49:0/570405806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31001577d0 con 0x7f3100072660 2024-04-15T15:18:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.134+0000 7f30ffe00640 1 -- 172.21.15.49:0/570405806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3100072160 con 0x7f3100072660 2024-04-15T15:18:49.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.135+0000 7f30f6a00640 1 -- 172.21.15.49:0/570405806 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30f0019020 con 0x7f3100072660 2024-04-15T15:18:49.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.136+0000 7f30f6a00640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30e006c8f0 0x7f30e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.136+0000 7f30f6a00640 1 -- 172.21.15.49:0/570405806 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f30f008b8f0 con 0x7f3100072660 2024-04-15T15:18:49.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.136+0000 7f30fe000640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30e006c8f0 0x7f30e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.137+0000 7f30fe000640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30e006c8f0 0x7f30e006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30ec036000 tx=0x7f30ec03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.141+0000 7f30f6a00640 1 -- 172.21.15.49:0/570405806 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30f0020070 con 0x7f3100072660 2024-04-15T15:18:49.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.387+0000 7f30ffe00640 1 -- 172.21.15.49:0/570405806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3100160c00 con 0x7f3100072660 2024-04-15T15:18:49.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.389+0000 7f30f6a00640 1 -- 172.21.15.49:0/570405806 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f30f0059c80 con 0x7f3100072660 2024-04-15T15:18:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:49.391 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/203558139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":203558139},{"type":"v1","addr":"172.21.15.107:6833","nonce":203558139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3198352923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3198352923},{"type":"v1","addr":"172.21.15.49:6836","nonce":3198352923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1425337425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1425337425},{"type":"v1","addr":"172.21.15.49:6837","nonce":1425337425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:18:45.325981+0000","modified":"2024-04-15T15:18:48.378286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2374398161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2374398161},{"type":"v1","addr":"172.21.15.107:6835","nonce":2374398161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-15T15:18:49.392 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-04-15T15:18:49.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.391+0000 7f30ffe00640 1 -- 172.21.15.49:0/570405806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30e006c8f0 msgr2=0x7f30e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:49.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.391+0000 7f30ffe00640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30e006c8f0 0x7f30e006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30ec036000 tx=0x7f30ec03c040 comp rx=0 tx=0).stop 2024-04-15T15:18:49.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.391+0000 7f30ffe00640 1 -- 172.21.15.49:0/570405806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 msgr2=0x7f310014e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.392+0000 7f30ffe00640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 0x7f310014e6b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f30f000c7c0 tx=0x7f30f000cc80 comp rx=0 tx=0).stop 2024-04-15T15:18:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.392+0000 7f30ffe00640 1 -- 172.21.15.49:0/570405806 shutdown_connections 2024-04-15T15:18:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.392+0000 7f30ffe00640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f30e006c8f0 0x7f30e006edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.392+0000 7f30ffe00640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f310015cbe0 0x7f310015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.392+0000 7f30ffe00640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31000cee00 0x7f310015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.392+0000 7f30ffe00640 1 --2- 172.21.15.49:0/570405806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3100072660 0x7f310014e6b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.392+0000 7f30ffe00640 1 -- 172.21.15.49:0/570405806 >> 172.21.15.49:0/570405806 conn(0x7f310006b930 msgr2=0x7f3100074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.392+0000 7f30ffe00640 1 -- 172.21.15.49:0/570405806 shutdown_connections 2024-04-15T15:18:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.392+0000 7f30ffe00640 1 -- 172.21.15.49:0/570405806 wait complete. 2024-04-15T15:18:49.415 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:49.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.643+0000 7fa185e00640 1 Processor -- start 2024-04-15T15:18:49.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.643+0000 7fa185e00640 1 -- start start 2024-04-15T15:18:49.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.645+0000 7fa185e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa180072660 0x7fa18006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.645+0000 7fa185e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa18015cbe0 0x7fa18015d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.645+0000 7fa185e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa18015dbb0 con 0x7fa1800cee00 2024-04-15T15:18:49.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.645+0000 7fa185e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fa18006f4f0 con 0x7fa180072660 2024-04-15T15:18:49.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.645+0000 7fa185e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fa18015d8d0 con 0x7fa18015cbe0 2024-04-15T15:18:49.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.646+0000 7fa177e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa180072660 0x7fa18006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.646+0000 7fa177e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa180072660 0x7fa18006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58296/0 (socket says 172.21.15.49:58296) 2024-04-15T15:18:49.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.646+0000 7fa177e00640 1 -- 172.21.15.49:0/3895419714 learned_addr learned my addr 172.21.15.49:0/3895419714 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:49.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.646+0000 7fa185400640 1 --2- 172.21.15.49:0/3895419714 >> v2:172.21.15.107:3300/0 conn(0x7fa18015cbe0 0x7fa18015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.646+0000 7fa177e00640 1 -- 172.21.15.49:0/3895419714 >> v2:172.21.15.107:3300/0 conn(0x7fa18015cbe0 msgr2=0x7fa18015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:49.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.646+0000 7fa177e00640 1 --2- 172.21.15.49:0/3895419714 >> v2:172.21.15.107:3300/0 conn(0x7fa18015cbe0 0x7fa18015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.646+0000 7fa177e00640 1 -- 172.21.15.49:0/3895419714 >> v1:172.21.15.107:6789/0 conn(0x7fa1800cee00 legacy=0x7fa180071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:49.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.646+0000 7fa177e00640 1 -- 172.21.15.49:0/3895419714 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18006dde0 con 0x7fa180072660 2024-04-15T15:18:49.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.647+0000 7fa177e00640 1 --2- 172.21.15.49:0/3895419714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180072660 0x7fa18006e8c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1800b02a0 tx=0x7fa16802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e048a682023ec90 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:49.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.648+0000 7fa17fe00640 1 -- 172.21.15.49:0/3895419714 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa168035020 con 0x7fa180072660 2024-04-15T15:18:49.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.648+0000 7fa17fe00640 1 -- 172.21.15.49:0/3895419714 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa168031870 con 0x7fa180072660 2024-04-15T15:18:49.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.648+0000 7fa17fe00640 1 -- 172.21.15.49:0/3895419714 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa168031c40 con 0x7fa180072660 2024-04-15T15:18:49.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.648+0000 7fa185e00640 1 -- 172.21.15.49:0/3895419714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180072660 msgr2=0x7fa18006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:49.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.648+0000 7fa185e00640 1 --2- 172.21.15.49:0/3895419714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180072660 0x7fa18006e8c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1800b02a0 tx=0x7fa16802b910 comp rx=0 tx=0).stop 2024-04-15T15:18:49.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.648+0000 7fa185e00640 1 -- 172.21.15.49:0/3895419714 shutdown_connections 2024-04-15T15:18:49.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.649+0000 7fa185e00640 1 --2- 172.21.15.49:0/3895419714 >> v2:172.21.15.107:3300/0 conn(0x7fa18015cbe0 0x7fa18015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.649+0000 7fa185e00640 1 --2- 172.21.15.49:0/3895419714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180072660 0x7fa18006e8c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.649+0000 7fa185e00640 1 -- 172.21.15.49:0/3895419714 >> 172.21.15.49:0/3895419714 conn(0x7fa18006b930 msgr2=0x7fa18006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:49.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.649+0000 7fa185e00640 1 -- 172.21.15.49:0/3895419714 shutdown_connections 2024-04-15T15:18:49.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.649+0000 7fa185e00640 1 -- 172.21.15.49:0/3895419714 wait complete. 2024-04-15T15:18:49.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.650+0000 7fa185e00640 1 Processor -- start 2024-04-15T15:18:49.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.651+0000 7fa185e00640 1 -- start start 2024-04-15T15:18:49.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.651+0000 7fa185e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa180072660 0x7fa18014e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1800cee00 0x7fa180152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015cbe0 0x7fa180154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa18015d5b0 con 0x7fa18015cbe0 2024-04-15T15:18:49.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa18006e3a0 con 0x7fa180072660 2024-04-15T15:18:49.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa18015dda0 con 0x7fa1800cee00 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015cbe0 0x7fa180154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015cbe0 0x7fa180154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58308/0 (socket says 172.21.15.49:58308) 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185400640 1 -- 172.21.15.49:0/2285795625 learned_addr learned my addr 172.21.15.49:0/2285795625 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa177e00640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1800cee00 0x7fa180152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa184a00640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa180072660 0x7fa18014e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185400640 1 -- 172.21.15.49:0/2285795625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1800cee00 msgr2=0x7fa180152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185400640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1800cee00 0x7fa180152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185400640 1 -- 172.21.15.49:0/2285795625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa180072660 msgr2=0x7fa18014e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185400640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa180072660 0x7fa18014e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.652+0000 7fa185400640 1 -- 172.21.15.49:0/2285795625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa180159940 con 0x7fa18015cbe0 2024-04-15T15:18:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.653+0000 7fa185400640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015cbe0 0x7fa180154070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa178007e60 tx=0x7fa17800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:49.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.653+0000 7fa17d600640 1 -- 172.21.15.49:0/2285795625 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178014420 con 0x7fa18015cbe0 2024-04-15T15:18:49.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.653+0000 7fa17d600640 1 -- 172.21.15.49:0/2285795625 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa178019070 con 0x7fa18015cbe0 2024-04-15T15:18:49.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.653+0000 7fa17d600640 1 -- 172.21.15.49:0/2285795625 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178014820 con 0x7fa18015cbe0 2024-04-15T15:18:49.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.653+0000 7fa185e00640 1 -- 172.21.15.49:0/2285795625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa180152cc0 con 0x7fa18015cbe0 2024-04-15T15:18:49.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.653+0000 7fa185e00640 1 -- 172.21.15.49:0/2285795625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa180158e60 con 0x7fa18015cbe0 2024-04-15T15:18:49.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.654+0000 7fa185e00640 1 -- 172.21.15.49:0/2285795625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18006bfd0 con 0x7fa18015cbe0 2024-04-15T15:18:49.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.655+0000 7fa17d600640 1 -- 172.21.15.49:0/2285795625 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1780149c0 con 0x7fa18015cbe0 2024-04-15T15:18:49.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.657+0000 7fa17d600640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa15c06c940 0x7fa15c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:49.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.657+0000 7fa17d600640 1 -- 172.21.15.49:0/2285795625 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa17808c2e0 con 0x7fa18015cbe0 2024-04-15T15:18:49.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.657+0000 7fa184a00640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa15c06c940 0x7fa15c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:49.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.658+0000 7fa184a00640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa15c06c940 0x7fa15c06ee00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa170004ea0 tx=0x7fa170009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:49.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.661+0000 7fa17d600640 1 -- 172.21.15.49:0/2285795625 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa17805a7f0 con 0x7fa18015cbe0 2024-04-15T15:18:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.867+0000 7fa185e00640 1 -- 172.21.15.49:0/2285795625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa180156280 con 0x7fa18015cbe0 2024-04-15T15:18:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.870+0000 7fa17d600640 1 -- 172.21.15.49:0/2285795625 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7fa17805a0f0 con 0x7fa18015cbe0 2024-04-15T15:18:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:49.872 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.873+0000 7fa185e00640 1 -- 172.21.15.49:0/2285795625 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa15c06c940 msgr2=0x7fa15c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.873+0000 7fa185e00640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa15c06c940 0x7fa15c06ee00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa170004ea0 tx=0x7fa170009040 comp rx=0 tx=0).stop 2024-04-15T15:18:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.873+0000 7fa185e00640 1 -- 172.21.15.49:0/2285795625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015cbe0 msgr2=0x7fa180154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.873+0000 7fa185e00640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015cbe0 0x7fa180154070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa178007e60 tx=0x7fa17800e4b0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.873+0000 7fa185e00640 1 -- 172.21.15.49:0/2285795625 shutdown_connections 2024-04-15T15:18:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.873+0000 7fa185e00640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa15c06c940 0x7fa15c06ee00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.873+0000 7fa185e00640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1800cee00 0x7fa180152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.873+0000 7fa185e00640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015cbe0 0x7fa180154070 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.873+0000 7fa185e00640 1 --2- 172.21.15.49:0/2285795625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa180072660 0x7fa18014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:49.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.873+0000 7fa185e00640 1 -- 172.21.15.49:0/2285795625 >> 172.21.15.49:0/2285795625 conn(0x7fa18006b930 msgr2=0x7fa180156af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:49.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.874+0000 7fa185e00640 1 -- 172.21.15.49:0/2285795625 shutdown_connections 2024-04-15T15:18:49.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:49.874+0000 7fa185e00640 1 -- 172.21.15.49:0/2285795625 wait complete. 2024-04-15T15:18:49.894 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-15T15:18:49.894 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:18:50.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.110+0000 7f7e13200640 1 Processor -- start 2024-04-15T15:18:50.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.110+0000 7f7e13200640 1 -- start start 2024-04-15T15:18:50.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.111+0000 7f7e13200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e14162020 0x7f7e14162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.111+0000 7f7e13200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7e140d1c90 0x7f7e140d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.111+0000 7f7e13200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e140d0090 con 0x7f7e140cf110 2024-04-15T15:18:50.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.111+0000 7f7e13200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f7e14163440 con 0x7f7e140d1c90 2024-04-15T15:18:50.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.111+0000 7f7e13200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7e140b7b30 con 0x7f7e14162020 2024-04-15T15:18:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.112+0000 7f7e11e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e14162020 0x7f7e14162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.112+0000 7f7e12800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7e140d1c90 0x7f7e140d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.112+0000 7f7e11400640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f7e140cf110 0x7f7e140d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:46228/0 (socket says 172.21.15.49:46228) 2024-04-15T15:18:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.112+0000 7f7e11400640 1 -- 172.21.15.49:0/2394818574 learned_addr learned my addr 172.21.15.49:0/2394818574 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.112+0000 7f7e11e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e14162020 0x7f7e14162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:45868/0 (socket says 172.21.15.49:45868) 2024-04-15T15:18:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.113+0000 7f7e12800640 1 -- 172.21.15.49:0/2394818574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e14162020 msgr2=0x7f7e14162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.113+0000 7f7e12800640 1 --2- 172.21.15.49:0/2394818574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e14162020 0x7f7e14162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.113+0000 7f7e12800640 1 -- 172.21.15.49:0/2394818574 >> v1:172.21.15.107:6789/0 conn(0x7f7e140cf110 legacy=0x7f7e140d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.113+0000 7f7e12800640 1 -- 172.21.15.49:0/2394818574 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e140cfad0 con 0x7f7e140d1c90 2024-04-15T15:18:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.115+0000 7f7e12800640 1 --2- 172.21.15.49:0/2394818574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e140d1c90 0x7f7e140d3390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009d90 tx=0x7f7e0c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9cadde88436b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:50.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.115+0000 7f7e10a00640 1 -- 172.21.15.49:0/2394818574 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c039070 con 0x7f7e140d1c90 2024-04-15T15:18:50.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.115+0000 7f7e10a00640 1 -- 172.21.15.49:0/2394818574 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0c02cc50 con 0x7f7e140d1c90 2024-04-15T15:18:50.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.115+0000 7f7e10a00640 1 -- 172.21.15.49:0/2394818574 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c034440 con 0x7f7e140d1c90 2024-04-15T15:18:50.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.116+0000 7f7e13200640 1 -- 172.21.15.49:0/2394818574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e140d1c90 msgr2=0x7f7e140d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.116+0000 7f7e13200640 1 --2- 172.21.15.49:0/2394818574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e140d1c90 0x7f7e140d3390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009d90 tx=0x7f7e0c02bea0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.116+0000 7f7e13200640 1 -- 172.21.15.49:0/2394818574 shutdown_connections 2024-04-15T15:18:50.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.116+0000 7f7e13200640 1 --2- 172.21.15.49:0/2394818574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e140d1c90 0x7f7e140d3390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.116+0000 7f7e13200640 1 --2- 172.21.15.49:0/2394818574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e14162020 0x7f7e14162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.116+0000 7f7e13200640 1 -- 172.21.15.49:0/2394818574 >> 172.21.15.49:0/2394818574 conn(0x7f7e1406cf00 msgr2=0x7f7e1406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:50.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.116+0000 7f7e13200640 1 -- 172.21.15.49:0/2394818574 shutdown_connections 2024-04-15T15:18:50.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.117+0000 7f7e13200640 1 -- 172.21.15.49:0/2394818574 wait complete. 2024-04-15T15:18:50.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.119+0000 7f7e13200640 1 Processor -- start 2024-04-15T15:18:50.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.119+0000 7f7e13200640 1 -- start start 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.120+0000 7f7e13200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e140cf110 0x7f7e14155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.121+0000 7f7e13200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e140d1c90 0x7f7e14155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.121+0000 7f7e13200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e14162020 0x7f7e14156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.121+0000 7f7e13200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7e140d2b90 con 0x7f7e14162020 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.121+0000 7f7e13200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e140cf880 con 0x7f7e140cf110 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.121+0000 7f7e13200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7e140d2e70 con 0x7f7e140d1c90 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.121+0000 7f7e12800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e14162020 0x7f7e14156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.121+0000 7f7e12800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e14162020 0x7f7e14156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58314/0 (socket says 172.21.15.49:58314) 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.121+0000 7f7e12800640 1 -- 172.21.15.49:0/1920411905 learned_addr learned my addr 172.21.15.49:0/1920411905 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.121+0000 7f7e11e00640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e140cf110 0x7f7e14155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.121+0000 7f7e11400640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e140d1c90 0x7f7e14155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.122+0000 7f7e12800640 1 -- 172.21.15.49:0/1920411905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e140d1c90 msgr2=0x7f7e14155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.122+0000 7f7e12800640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e140d1c90 0x7f7e14155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.122+0000 7f7e12800640 1 -- 172.21.15.49:0/1920411905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e140cf110 msgr2=0x7f7e14155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.122+0000 7f7e12800640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e140cf110 0x7f7e14155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.122+0000 7f7e12800640 1 -- 172.21.15.49:0/1920411905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e141f48d0 con 0x7f7e14162020 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.122+0000 7f7e12800640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e14162020 0x7f7e14156150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009eb0 tx=0x7f7e0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.122+0000 7f7e0a000640 1 -- 172.21.15.49:0/1920411905 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c044020 con 0x7f7e14162020 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.122+0000 7f7e0a000640 1 -- 172.21.15.49:0/1920411905 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0c02ca50 con 0x7f7e14162020 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.122+0000 7f7e0a000640 1 -- 172.21.15.49:0/1920411905 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c039040 con 0x7f7e14162020 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.122+0000 7f7e13200640 1 -- 172.21.15.49:0/1920411905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e141f4b90 con 0x7f7e14162020 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.123+0000 7f7e13200640 1 -- 172.21.15.49:0/1920411905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e141f5150 con 0x7f7e14162020 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.124+0000 7f7e13200640 1 -- 172.21.15.49:0/1920411905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e140755c0 con 0x7f7e14162020 2024-04-15T15:18:50.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.125+0000 7f7e0a000640 1 -- 172.21.15.49:0/1920411905 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e0c047020 con 0x7f7e14162020 2024-04-15T15:18:50.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.126+0000 7f7e0a000640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7df406c8f0 0x7f7df406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.126+0000 7f7e0a000640 1 -- 172.21.15.49:0/1920411905 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7e0c0ae380 con 0x7f7e14162020 2024-04-15T15:18:50.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.126+0000 7f7e11e00640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7df406c8f0 0x7f7df406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.127+0000 7f7e11e00640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7df406c8f0 0x7f7df406edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009b40 tx=0x7f7e04009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.130+0000 7f7e0a000640 1 -- 172.21.15.49:0/1920411905 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e0c04e030 con 0x7f7e14162020 2024-04-15T15:18:50.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.338+0000 7f7e13200640 1 -- 172.21.15.49:0/1920411905 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e140d25a0 con 0x7f7df406c8f0 2024-04-15T15:18:50.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.347+0000 7f7e0a000640 1 -- 172.21.15.49:0/1920411905 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f7e140d25a0 con 0x7f7df406c8f0 2024-04-15T15:18:50.353 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:50.354 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:18:50.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.352+0000 7f7e13200640 1 -- 172.21.15.49:0/1920411905 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7df406c8f0 msgr2=0x7f7df406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7df406c8f0 0x7f7df406edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009b40 tx=0x7f7e04009190 comp rx=0 tx=0).stop 2024-04-15T15:18:50.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 -- 172.21.15.49:0/1920411905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e14162020 msgr2=0x7f7e14156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e14162020 0x7f7e14156150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009eb0 tx=0x7f7e0c004a40 comp rx=0 tx=0).stop 2024-04-15T15:18:50.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 -- 172.21.15.49:0/1920411905 shutdown_connections 2024-04-15T15:18:50.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7df406c8f0 0x7f7df406edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e140d1c90 0x7f7e14155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e14162020 0x7f7e14156150 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 --2- 172.21.15.49:0/1920411905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e140cf110 0x7f7e14155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 -- 172.21.15.49:0/1920411905 >> 172.21.15.49:0/1920411905 conn(0x7f7e1406cf00 msgr2=0x7f7e140d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:50.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 -- 172.21.15.49:0/1920411905 shutdown_connections 2024-04-15T15:18:50.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.353+0000 7f7e13200640 1 -- 172.21.15.49:0/1920411905 wait complete. 2024-04-15T15:18:50.373 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-15T15:18:48.771378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236432,"kb_used_data":2900,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738288,"statfs":{"total":773094113280,"available":772852006912,"internally_reserved":0,"allocated":2969600,"data_stored":1772720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":62,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.952817"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.825719+0000","last_change":"2024-04-15T15:18:45.850244+0000","last_active":"2024-04-15T15:18:46.825719+0000","last_peered":"2024-04-15T15:18:46.825719+0000","last_clean":"2024-04-15T15:18:46.825719+0000","last_became_active":"2024-04-15T15:18:45.849857+0000","last_became_peered":"2024-04-15T15:18:45.849857+0000","last_unstale":"2024-04-15T15:18:46.825719+0000","last_undegraded":"2024-04-15T15:18:46.825719+0000","last_fullsized":"2024-04-15T15:18:46.825719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:08:04.293347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870331+0000","last_change":"2024-04-15T15:18:45.843425+0000","last_active":"2024-04-15T15:18:46.870331+0000","last_peered":"2024-04-15T15:18:46.870331+0000","last_clean":"2024-04-15T15:18:46.870331+0000","last_became_active":"2024-04-15T15:18:45.843052+0000","last_became_peered":"2024-04-15T15:18:45.843052+0000","last_unstale":"2024-04-15T15:18:46.870331+0000","last_undegraded":"2024-04-15T15:18:46.870331+0000","last_fullsized":"2024-04-15T15:18:46.870331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.860635+0000","last_change":"2024-04-15T15:18:45.853920+0000","last_active":"2024-04-15T15:18:45.860635+0000","last_peered":"2024-04-15T15:18:45.860635+0000","last_clean":"2024-04-15T15:18:45.860635+0000","last_became_active":"2024-04-15T15:18:45.853659+0000","last_became_peered":"2024-04-15T15:18:45.853659+0000","last_unstale":"2024-04-15T15:18:45.860635+0000","last_undegraded":"2024-04-15T15:18:45.860635+0000","last_fullsized":"2024-04-15T15:18:45.860635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:27.690982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823670+0000","last_change":"2024-04-15T15:18:45.861695+0000","last_active":"2024-04-15T15:18:46.823670+0000","last_peered":"2024-04-15T15:18:46.823670+0000","last_clean":"2024-04-15T15:18:46.823670+0000","last_became_active":"2024-04-15T15:18:45.858560+0000","last_became_peered":"2024-04-15T15:18:45.858560+0000","last_unstale":"2024-04-15T15:18:46.823670+0000","last_undegraded":"2024-04-15T15:18:46.823670+0000","last_fullsized":"2024-04-15T15:18:46.823670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:29:14.132066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820061+0000","last_change":"2024-04-15T15:18:45.849530+0000","last_active":"2024-04-15T15:18:46.820061+0000","last_peered":"2024-04-15T15:18:46.820061+0000","last_clean":"2024-04-15T15:18:46.820061+0000","last_became_active":"2024-04-15T15:18:45.849255+0000","last_became_peered":"2024-04-15T15:18:45.849255+0000","last_unstale":"2024-04-15T15:18:46.820061+0000","last_undegraded":"2024-04-15T15:18:46.820061+0000","last_fullsized":"2024-04-15T15:18:46.820061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:36:03.614062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822193+0000","last_change":"2024-04-15T15:18:45.842283+0000","last_active":"2024-04-15T15:18:46.822193+0000","last_peered":"2024-04-15T15:18:46.822193+0000","last_clean":"2024-04-15T15:18:46.822193+0000","last_became_active":"2024-04-15T15:18:45.842004+0000","last_became_peered":"2024-04-15T15:18:45.842004+0000","last_unstale":"2024-04-15T15:18:46.822193+0000","last_undegraded":"2024-04-15T15:18:46.822193+0000","last_fullsized":"2024-04-15T15:18:46.822193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:08:29.977812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818174+0000","last_change":"2024-04-15T15:18:45.856124+0000","last_active":"2024-04-15T15:18:46.818174+0000","last_peered":"2024-04-15T15:18:46.818174+0000","last_clean":"2024-04-15T15:18:46.818174+0000","last_became_active":"2024-04-15T15:18:45.849034+0000","last_became_peered":"2024-04-15T15:18:45.849034+0000","last_unstale":"2024-04-15T15:18:46.818174+0000","last_undegraded":"2024-04-15T15:18:46.818174+0000","last_fullsized":"2024-04-15T15:18:46.818174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:49:43.342170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.855348+0000","last_change":"2024-04-15T15:18:45.848306+0000","last_active":"2024-04-15T15:18:45.855348+0000","last_peered":"2024-04-15T15:18:45.855348+0000","last_clean":"2024-04-15T15:18:45.855348+0000","last_became_active":"2024-04-15T15:18:45.848033+0000","last_became_peered":"2024-04-15T15:18:45.848033+0000","last_unstale":"2024-04-15T15:18:45.855348+0000","last_undegraded":"2024-04-15T15:18:45.855348+0000","last_fullsized":"2024-04-15T15:18:45.855348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:35.931556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.826189+0000","last_change":"2024-04-15T15:18:45.862452+0000","last_active":"2024-04-15T15:18:46.826189+0000","last_peered":"2024-04-15T15:18:46.826189+0000","last_clean":"2024-04-15T15:18:46.826189+0000","last_became_active":"2024-04-15T15:18:45.862157+0000","last_became_peered":"2024-04-15T15:18:45.862157+0000","last_unstale":"2024-04-15T15:18:46.826189+0000","last_undegraded":"2024-04-15T15:18:46.826189+0000","last_fullsized":"2024-04-15T15:18:46.826189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:14:17.068828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875290+0000","last_change":"2024-04-15T15:18:45.855709+0000","last_active":"2024-04-15T15:18:46.875290+0000","last_peered":"2024-04-15T15:18:46.875290+0000","last_clean":"2024-04-15T15:18:46.875290+0000","last_became_active":"2024-04-15T15:18:45.855243+0000","last_became_peered":"2024-04-15T15:18:45.855243+0000","last_unstale":"2024-04-15T15:18:46.875290+0000","last_undegraded":"2024-04-15T15:18:46.875290+0000","last_fullsized":"2024-04-15T15:18:46.875290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874937+0000","last_change":"2024-04-15T15:18:45.859684+0000","last_active":"2024-04-15T15:18:46.874937+0000","last_peered":"2024-04-15T15:18:46.874937+0000","last_clean":"2024-04-15T15:18:46.874937+0000","last_became_active":"2024-04-15T15:18:45.856683+0000","last_became_peered":"2024-04-15T15:18:45.856683+0000","last_unstale":"2024-04-15T15:18:46.874937+0000","last_undegraded":"2024-04-15T15:18:46.874937+0000","last_fullsized":"2024-04-15T15:18:46.874937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822415+0000","last_change":"2024-04-15T15:18:45.849769+0000","last_active":"2024-04-15T15:18:46.822415+0000","last_peered":"2024-04-15T15:18:46.822415+0000","last_clean":"2024-04-15T15:18:46.822415+0000","last_became_active":"2024-04-15T15:18:45.849462+0000","last_became_peered":"2024-04-15T15:18:45.849462+0000","last_unstale":"2024-04-15T15:18:46.822415+0000","last_undegraded":"2024-04-15T15:18:46.822415+0000","last_fullsized":"2024-04-15T15:18:46.822415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:35:53.211120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858253+0000","last_change":"2024-04-15T15:18:45.852551+0000","last_active":"2024-04-15T15:18:45.858253+0000","last_peered":"2024-04-15T15:18:45.858253+0000","last_clean":"2024-04-15T15:18:45.858253+0000","last_became_active":"2024-04-15T15:18:45.851658+0000","last_became_peered":"2024-04-15T15:18:45.851658+0000","last_unstale":"2024-04-15T15:18:45.858253+0000","last_undegraded":"2024-04-15T15:18:45.858253+0000","last_fullsized":"2024-04-15T15:18:45.858253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:27.690982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.849914+0000","last_change":"2024-04-15T15:18:45.842127+0000","last_active":"2024-04-15T15:18:45.849914+0000","last_peered":"2024-04-15T15:18:45.849914+0000","last_clean":"2024-04-15T15:18:45.849914+0000","last_became_active":"2024-04-15T15:18:45.841662+0000","last_became_peered":"2024-04-15T15:18:45.841662+0000","last_unstale":"2024-04-15T15:18:45.849914+0000","last_undegraded":"2024-04-15T15:18:45.849914+0000","last_fullsized":"2024-04-15T15:18:45.849914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:22.780040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869891+0000","last_change":"2024-04-15T15:18:45.832097+0000","last_active":"2024-04-15T15:18:46.869891+0000","last_peered":"2024-04-15T15:18:46.869891+0000","last_clean":"2024-04-15T15:18:46.869891+0000","last_became_active":"2024-04-15T15:18:45.831411+0000","last_became_peered":"2024-04-15T15:18:45.831411+0000","last_unstale":"2024-04-15T15:18:46.869891+0000","last_undegraded":"2024-04-15T15:18:46.869891+0000","last_fullsized":"2024-04-15T15:18:46.869891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822037+0000","last_change":"2024-04-15T15:18:45.846022+0000","last_active":"2024-04-15T15:18:46.822037+0000","last_peered":"2024-04-15T15:18:46.822037+0000","last_clean":"2024-04-15T15:18:46.822037+0000","last_became_active":"2024-04-15T15:18:45.844025+0000","last_became_peered":"2024-04-15T15:18:45.844025+0000","last_unstale":"2024-04-15T15:18:46.822037+0000","last_undegraded":"2024-04-15T15:18:46.822037+0000","last_fullsized":"2024-04-15T15:18:46.822037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.825219+0000","last_change":"2024-04-15T15:18:45.849397+0000","last_active":"2024-04-15T15:18:46.825219+0000","last_peered":"2024-04-15T15:18:46.825219+0000","last_clean":"2024-04-15T15:18:46.825219+0000","last_became_active":"2024-04-15T15:18:45.849114+0000","last_became_peered":"2024-04-15T15:18:45.849114+0000","last_unstale":"2024-04-15T15:18:46.825219+0000","last_undegraded":"2024-04-15T15:18:46.825219+0000","last_fullsized":"2024-04-15T15:18:46.825219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:08:04.293347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.850585+0000","last_change":"2024-04-15T15:18:45.848118+0000","last_active":"2024-04-15T15:18:45.850585+0000","last_peered":"2024-04-15T15:18:45.850585+0000","last_clean":"2024-04-15T15:18:45.850585+0000","last_became_active":"2024-04-15T15:18:45.847644+0000","last_became_peered":"2024-04-15T15:18:45.847644+0000","last_unstale":"2024-04-15T15:18:45.850585+0000","last_undegraded":"2024-04-15T15:18:45.850585+0000","last_fullsized":"2024-04-15T15:18:45.850585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:51.536346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820072+0000","last_change":"2024-04-15T15:18:45.860524+0000","last_active":"2024-04-15T15:18:46.820072+0000","last_peered":"2024-04-15T15:18:46.820072+0000","last_clean":"2024-04-15T15:18:46.820072+0000","last_became_active":"2024-04-15T15:18:45.852020+0000","last_became_peered":"2024-04-15T15:18:45.852020+0000","last_unstale":"2024-04-15T15:18:46.820072+0000","last_undegraded":"2024-04-15T15:18:46.820072+0000","last_fullsized":"2024-04-15T15:18:46.820072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:06:25.954239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874881+0000","last_change":"2024-04-15T15:18:45.852135+0000","last_active":"2024-04-15T15:18:46.874881+0000","last_peered":"2024-04-15T15:18:46.874881+0000","last_clean":"2024-04-15T15:18:46.874881+0000","last_became_active":"2024-04-15T15:18:45.851841+0000","last_became_peered":"2024-04-15T15:18:45.851841+0000","last_unstale":"2024-04-15T15:18:46.874881+0000","last_undegraded":"2024-04-15T15:18:46.874881+0000","last_fullsized":"2024-04-15T15:18:46.874881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.861379+0000","last_change":"2024-04-15T15:18:45.859189+0000","last_active":"2024-04-15T15:18:45.861379+0000","last_peered":"2024-04-15T15:18:45.861379+0000","last_clean":"2024-04-15T15:18:45.861379+0000","last_became_active":"2024-04-15T15:18:45.857117+0000","last_became_peered":"2024-04-15T15:18:45.857117+0000","last_unstale":"2024-04-15T15:18:45.861379+0000","last_undegraded":"2024-04-15T15:18:45.861379+0000","last_fullsized":"2024-04-15T15:18:45.861379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:57:54.115969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822607+0000","last_change":"2024-04-15T15:18:45.840913+0000","last_active":"2024-04-15T15:18:46.822607+0000","last_peered":"2024-04-15T15:18:46.822607+0000","last_clean":"2024-04-15T15:18:46.822607+0000","last_became_active":"2024-04-15T15:18:45.840257+0000","last_became_peered":"2024-04-15T15:18:45.840257+0000","last_unstale":"2024-04-15T15:18:46.822607+0000","last_undegraded":"2024-04-15T15:18:46.822607+0000","last_fullsized":"2024-04-15T15:18:46.822607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:49:43.342170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.857965+0000","last_change":"2024-04-15T15:18:45.851064+0000","last_active":"2024-04-15T15:18:45.857965+0000","last_peered":"2024-04-15T15:18:45.857965+0000","last_clean":"2024-04-15T15:18:45.857965+0000","last_became_active":"2024-04-15T15:18:45.850725+0000","last_became_peered":"2024-04-15T15:18:45.850725+0000","last_unstale":"2024-04-15T15:18:45.857965+0000","last_undegraded":"2024-04-15T15:18:45.857965+0000","last_fullsized":"2024-04-15T15:18:45.857965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:22.780040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820869+0000","last_change":"2024-04-15T15:18:45.838476+0000","last_active":"2024-04-15T15:18:46.820869+0000","last_peered":"2024-04-15T15:18:46.820869+0000","last_clean":"2024-04-15T15:18:46.820869+0000","last_became_active":"2024-04-15T15:18:45.838160+0000","last_became_peered":"2024-04-15T15:18:45.838160+0000","last_unstale":"2024-04-15T15:18:46.820869+0000","last_undegraded":"2024-04-15T15:18:46.820869+0000","last_fullsized":"2024-04-15T15:18:46.820869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858227+0000","last_change":"2024-04-15T15:18:45.855537+0000","last_active":"2024-04-15T15:18:45.858227+0000","last_peered":"2024-04-15T15:18:45.858227+0000","last_clean":"2024-04-15T15:18:45.858227+0000","last_became_active":"2024-04-15T15:18:45.855144+0000","last_became_peered":"2024-04-15T15:18:45.855144+0000","last_unstale":"2024-04-15T15:18:45.858227+0000","last_undegraded":"2024-04-15T15:18:45.858227+0000","last_fullsized":"2024-04-15T15:18:45.858227+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:18.727426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875245+0000","last_change":"2024-04-15T15:18:45.848420+0000","last_active":"2024-04-15T15:18:46.875245+0000","last_peered":"2024-04-15T15:18:46.875245+0000","last_clean":"2024-04-15T15:18:46.875245+0000","last_became_active":"2024-04-15T15:18:45.848080+0000","last_became_peered":"2024-04-15T15:18:45.848080+0000","last_unstale":"2024-04-15T15:18:46.875245+0000","last_undegraded":"2024-04-15T15:18:46.875245+0000","last_fullsized":"2024-04-15T15:18:46.875245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869996+0000","last_change":"2024-04-15T15:18:45.858358+0000","last_active":"2024-04-15T15:18:46.869996+0000","last_peered":"2024-04-15T15:18:46.869996+0000","last_clean":"2024-04-15T15:18:46.869996+0000","last_became_active":"2024-04-15T15:18:45.856289+0000","last_became_peered":"2024-04-15T15:18:45.856289+0000","last_unstale":"2024-04-15T15:18:46.869996+0000","last_undegraded":"2024-04-15T15:18:46.869996+0000","last_fullsized":"2024-04-15T15:18:46.869996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858144+0000","last_change":"2024-04-15T15:18:45.854349+0000","last_active":"2024-04-15T15:18:45.858144+0000","last_peered":"2024-04-15T15:18:45.858144+0000","last_clean":"2024-04-15T15:18:45.858144+0000","last_became_active":"2024-04-15T15:18:45.853978+0000","last_became_peered":"2024-04-15T15:18:45.853978+0000","last_unstale":"2024-04-15T15:18:45.858144+0000","last_undegraded":"2024-04-15T15:18:45.858144+0000","last_fullsized":"2024-04-15T15:18:45.858144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:17:10.950382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.861076+0000","last_change":"2024-04-15T15:18:45.858661+0000","last_active":"2024-04-15T15:18:45.861076+0000","last_peered":"2024-04-15T15:18:45.861076+0000","last_clean":"2024-04-15T15:18:45.861076+0000","last_became_active":"2024-04-15T15:18:45.858244+0000","last_became_peered":"2024-04-15T15:18:45.858244+0000","last_unstale":"2024-04-15T15:18:45.861076+0000","last_undegraded":"2024-04-15T15:18:45.861076+0000","last_fullsized":"2024-04-15T15:18:45.861076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:17:10.950382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823029+0000","last_change":"2024-04-15T15:18:45.840590+0000","last_active":"2024-04-15T15:18:46.823029+0000","last_peered":"2024-04-15T15:18:46.823029+0000","last_clean":"2024-04-15T15:18:46.823029+0000","last_became_active":"2024-04-15T15:18:45.839849+0000","last_became_peered":"2024-04-15T15:18:45.839849+0000","last_unstale":"2024-04-15T15:18:46.823029+0000","last_undegraded":"2024-04-15T15:18:46.823029+0000","last_fullsized":"2024-04-15T15:18:46.823029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.566254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.850135+0000","last_change":"2024-04-15T15:18:45.847940+0000","last_active":"2024-04-15T15:18:45.850135+0000","last_peered":"2024-04-15T15:18:45.850135+0000","last_clean":"2024-04-15T15:18:45.850135+0000","last_became_active":"2024-04-15T15:18:45.846916+0000","last_became_peered":"2024-04-15T15:18:45.846916+0000","last_unstale":"2024-04-15T15:18:45.850135+0000","last_undegraded":"2024-04-15T15:18:45.850135+0000","last_fullsized":"2024-04-15T15:18:45.850135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:20.202506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874483+0000","last_change":"2024-04-15T15:18:45.846327+0000","last_active":"2024-04-15T15:18:46.874483+0000","last_peered":"2024-04-15T15:18:46.874483+0000","last_clean":"2024-04-15T15:18:46.874483+0000","last_became_active":"2024-04-15T15:18:45.846010+0000","last_became_peered":"2024-04-15T15:18:45.846010+0000","last_unstale":"2024-04-15T15:18:46.874483+0000","last_undegraded":"2024-04-15T15:18:46.874483+0000","last_fullsized":"2024-04-15T15:18:46.874483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820924+0000","last_change":"2024-04-15T15:18:45.846764+0000","last_active":"2024-04-15T15:18:46.820924+0000","last_peered":"2024-04-15T15:18:46.820924+0000","last_clean":"2024-04-15T15:18:46.820924+0000","last_became_active":"2024-04-15T15:18:45.842870+0000","last_became_peered":"2024-04-15T15:18:45.842870+0000","last_unstale":"2024-04-15T15:18:46.820924+0000","last_undegraded":"2024-04-15T15:18:46.820924+0000","last_fullsized":"2024-04-15T15:18:46.820924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:38:23.700029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.875822+0000","last_change":"2024-04-15T15:18:44.864457+0000","last_active":"2024-04-15T15:18:45.875822+0000","last_peered":"2024-04-15T15:18:45.875822+0000","last_clean":"2024-04-15T15:18:45.875822+0000","last_became_active":"2024-04-15T15:18:44.863999+0000","last_became_peered":"2024-04-15T15:18:44.863999+0000","last_unstale":"2024-04-15T15:18:45.875822+0000","last_undegraded":"2024-04-15T15:18:45.875822+0000","last_fullsized":"2024-04-15T15:18:45.875822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.861065+0000","last_change":"2024-04-15T15:18:45.855613+0000","last_active":"2024-04-15T15:18:45.861065+0000","last_peered":"2024-04-15T15:18:45.861065+0000","last_clean":"2024-04-15T15:18:45.861065+0000","last_became_active":"2024-04-15T15:18:45.855327+0000","last_became_peered":"2024-04-15T15:18:45.855327+0000","last_unstale":"2024-04-15T15:18:45.861065+0000","last_undegraded":"2024-04-15T15:18:45.861065+0000","last_fullsized":"2024-04-15T15:18:45.861065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:22:54.875123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874502+0000","last_change":"2024-04-15T15:18:44.864183+0000","last_active":"2024-04-15T15:18:46.874502+0000","last_peered":"2024-04-15T15:18:46.874502+0000","last_clean":"2024-04-15T15:18:46.874502+0000","last_became_active":"2024-04-15T15:18:44.861740+0000","last_became_peered":"2024-04-15T15:18:44.861740+0000","last_unstale":"2024-04-15T15:18:46.874502+0000","last_undegraded":"2024-04-15T15:18:46.874502+0000","last_fullsized":"2024-04-15T15:18:46.874502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.848791+0000","last_change":"2024-04-15T15:18:44.867636+0000","last_active":"2024-04-15T15:18:45.848791+0000","last_peered":"2024-04-15T15:18:45.848791+0000","last_clean":"2024-04-15T15:18:45.848791+0000","last_became_active":"2024-04-15T15:18:44.867307+0000","last_became_peered":"2024-04-15T15:18:44.867307+0000","last_unstale":"2024-04-15T15:18:45.848791+0000","last_undegraded":"2024-04-15T15:18:45.848791+0000","last_fullsized":"2024-04-15T15:18:45.848791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858064+0000","last_change":"2024-04-15T15:18:45.850873+0000","last_active":"2024-04-15T15:18:45.858064+0000","last_peered":"2024-04-15T15:18:45.858064+0000","last_clean":"2024-04-15T15:18:45.858064+0000","last_became_active":"2024-04-15T15:18:45.850532+0000","last_became_peered":"2024-04-15T15:18:45.850532+0000","last_unstale":"2024-04-15T15:18:45.858064+0000","last_undegraded":"2024-04-15T15:18:45.858064+0000","last_fullsized":"2024-04-15T15:18:45.858064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:22:54.875123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821134+0000","last_change":"2024-04-15T15:18:44.866286+0000","last_active":"2024-04-15T15:18:46.821134+0000","last_peered":"2024-04-15T15:18:46.821134+0000","last_clean":"2024-04-15T15:18:46.821134+0000","last_became_active":"2024-04-15T15:18:44.863387+0000","last_became_peered":"2024-04-15T15:18:44.863387+0000","last_unstale":"2024-04-15T15:18:46.821134+0000","last_undegraded":"2024-04-15T15:18:46.821134+0000","last_fullsized":"2024-04-15T15:18:46.821134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869851+0000","last_change":"2024-04-15T15:18:45.839256+0000","last_active":"2024-04-15T15:18:46.869851+0000","last_peered":"2024-04-15T15:18:46.869851+0000","last_clean":"2024-04-15T15:18:46.869851+0000","last_became_active":"2024-04-15T15:18:45.838960+0000","last_became_peered":"2024-04-15T15:18:45.838960+0000","last_unstale":"2024-04-15T15:18:46.869851+0000","last_undegraded":"2024-04-15T15:18:46.869851+0000","last_fullsized":"2024-04-15T15:18:46.869851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875122+0000","last_change":"2024-04-15T15:18:44.870228+0000","last_active":"2024-04-15T15:18:46.875122+0000","last_peered":"2024-04-15T15:18:46.875122+0000","last_clean":"2024-04-15T15:18:46.875122+0000","last_became_active":"2024-04-15T15:18:44.869155+0000","last_became_peered":"2024-04-15T15:18:44.869155+0000","last_unstale":"2024-04-15T15:18:46.875122+0000","last_undegraded":"2024-04-15T15:18:46.875122+0000","last_fullsized":"2024-04-15T15:18:46.875122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824756+0000","last_change":"2024-04-15T15:18:45.845321+0000","last_active":"2024-04-15T15:18:46.824756+0000","last_peered":"2024-04-15T15:18:46.824756+0000","last_clean":"2024-04-15T15:18:46.824756+0000","last_became_active":"2024-04-15T15:18:45.840422+0000","last_became_peered":"2024-04-15T15:18:45.840422+0000","last_unstale":"2024-04-15T15:18:46.824756+0000","last_undegraded":"2024-04-15T15:18:46.824756+0000","last_fullsized":"2024-04-15T15:18:46.824756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:32:58.852361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824852+0000","last_change":"2024-04-15T15:18:44.865670+0000","last_active":"2024-04-15T15:18:46.824852+0000","last_peered":"2024-04-15T15:18:46.824852+0000","last_clean":"2024-04-15T15:18:46.824852+0000","last_became_active":"2024-04-15T15:18:44.864795+0000","last_became_peered":"2024-04-15T15:18:44.864795+0000","last_unstale":"2024-04-15T15:18:46.824852+0000","last_undegraded":"2024-04-15T15:18:46.824852+0000","last_fullsized":"2024-04-15T15:18:46.824852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.855281+0000","last_change":"2024-04-15T15:18:45.852169+0000","last_active":"2024-04-15T15:18:45.855281+0000","last_peered":"2024-04-15T15:18:45.855281+0000","last_clean":"2024-04-15T15:18:45.855281+0000","last_became_active":"2024-04-15T15:18:45.845297+0000","last_became_peered":"2024-04-15T15:18:45.845297+0000","last_unstale":"2024-04-15T15:18:45.855281+0000","last_undegraded":"2024-04-15T15:18:45.855281+0000","last_fullsized":"2024-04-15T15:18:45.855281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:55.246279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870182+0000","last_change":"2024-04-15T15:18:44.866158+0000","last_active":"2024-04-15T15:18:46.870182+0000","last_peered":"2024-04-15T15:18:46.870182+0000","last_clean":"2024-04-15T15:18:46.870182+0000","last_became_active":"2024-04-15T15:18:44.865766+0000","last_became_peered":"2024-04-15T15:18:44.865766+0000","last_unstale":"2024-04-15T15:18:46.870182+0000","last_undegraded":"2024-04-15T15:18:46.870182+0000","last_fullsized":"2024-04-15T15:18:46.870182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822687+0000","last_change":"2024-04-15T15:18:45.843966+0000","last_active":"2024-04-15T15:18:46.822687+0000","last_peered":"2024-04-15T15:18:46.822687+0000","last_clean":"2024-04-15T15:18:46.822687+0000","last_became_active":"2024-04-15T15:18:45.842648+0000","last_became_peered":"2024-04-15T15:18:45.842648+0000","last_unstale":"2024-04-15T15:18:46.822687+0000","last_undegraded":"2024-04-15T15:18:46.822687+0000","last_fullsized":"2024-04-15T15:18:46.822687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:39:50.034981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874917+0000","last_change":"2024-04-15T15:18:44.867158+0000","last_active":"2024-04-15T15:18:46.874917+0000","last_peered":"2024-04-15T15:18:46.874917+0000","last_clean":"2024-04-15T15:18:46.874917+0000","last_became_active":"2024-04-15T15:18:44.866840+0000","last_became_peered":"2024-04-15T15:18:44.866840+0000","last_unstale":"2024-04-15T15:18:46.874917+0000","last_undegraded":"2024-04-15T15:18:46.874917+0000","last_fullsized":"2024-04-15T15:18:46.874917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818675+0000","last_change":"2024-04-15T15:18:45.860752+0000","last_active":"2024-04-15T15:18:46.818675+0000","last_peered":"2024-04-15T15:18:46.818675+0000","last_clean":"2024-04-15T15:18:46.818675+0000","last_became_active":"2024-04-15T15:18:45.860381+0000","last_became_peered":"2024-04-15T15:18:45.860381+0000","last_unstale":"2024-04-15T15:18:46.818675+0000","last_undegraded":"2024-04-15T15:18:46.818675+0000","last_fullsized":"2024-04-15T15:18:46.818675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:53.984837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.875312+0000","last_change":"2024-04-15T15:18:44.865018+0000","last_active":"2024-04-15T15:18:45.875312+0000","last_peered":"2024-04-15T15:18:45.875312+0000","last_clean":"2024-04-15T15:18:45.875312+0000","last_became_active":"2024-04-15T15:18:44.864073+0000","last_became_peered":"2024-04-15T15:18:44.864073+0000","last_unstale":"2024-04-15T15:18:45.875312+0000","last_undegraded":"2024-04-15T15:18:45.875312+0000","last_fullsized":"2024-04-15T15:18:45.875312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:34.796362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.861410+0000","last_change":"2024-04-15T15:18:45.860189+0000","last_active":"2024-04-15T15:18:45.861410+0000","last_peered":"2024-04-15T15:18:45.861410+0000","last_clean":"2024-04-15T15:18:45.861410+0000","last_became_active":"2024-04-15T15:18:45.859786+0000","last_became_peered":"2024-04-15T15:18:45.859786+0000","last_unstale":"2024-04-15T15:18:45.861410+0000","last_undegraded":"2024-04-15T15:18:45.861410+0000","last_fullsized":"2024-04-15T15:18:45.861410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:18.727426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822942+0000","last_change":"2024-04-15T15:18:44.866125+0000","last_active":"2024-04-15T15:18:46.822942+0000","last_peered":"2024-04-15T15:18:46.822942+0000","last_clean":"2024-04-15T15:18:46.822942+0000","last_became_active":"2024-04-15T15:18:44.864849+0000","last_became_peered":"2024-04-15T15:18:44.864849+0000","last_unstale":"2024-04-15T15:18:46.822942+0000","last_undegraded":"2024-04-15T15:18:46.822942+0000","last_fullsized":"2024-04-15T15:18:46.822942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819575+0000","last_change":"2024-04-15T15:18:45.852862+0000","last_active":"2024-04-15T15:18:46.819575+0000","last_peered":"2024-04-15T15:18:46.819575+0000","last_clean":"2024-04-15T15:18:46.819575+0000","last_became_active":"2024-04-15T15:18:45.851610+0000","last_became_peered":"2024-04-15T15:18:45.851610+0000","last_unstale":"2024-04-15T15:18:46.819575+0000","last_undegraded":"2024-04-15T15:18:46.819575+0000","last_fullsized":"2024-04-15T15:18:46.819575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:36.969295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874519+0000","last_change":"2024-04-15T15:18:44.871366+0000","last_active":"2024-04-15T15:18:46.874519+0000","last_peered":"2024-04-15T15:18:46.874519+0000","last_clean":"2024-04-15T15:18:46.874519+0000","last_became_active":"2024-04-15T15:18:44.870865+0000","last_became_peered":"2024-04-15T15:18:44.870865+0000","last_unstale":"2024-04-15T15:18:46.874519+0000","last_undegraded":"2024-04-15T15:18:46.874519+0000","last_fullsized":"2024-04-15T15:18:46.874519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874458+0000","last_change":"2024-04-15T15:18:45.847642+0000","last_active":"2024-04-15T15:18:46.874458+0000","last_peered":"2024-04-15T15:18:46.874458+0000","last_clean":"2024-04-15T15:18:46.874458+0000","last_became_active":"2024-04-15T15:18:45.845790+0000","last_became_peered":"2024-04-15T15:18:45.845790+0000","last_unstale":"2024-04-15T15:18:46.874458+0000","last_undegraded":"2024-04-15T15:18:46.874458+0000","last_fullsized":"2024-04-15T15:18:46.874458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.878253+0000","last_change":"2024-04-15T15:18:44.857029+0000","last_active":"2024-04-15T15:18:45.878253+0000","last_peered":"2024-04-15T15:18:45.878253+0000","last_clean":"2024-04-15T15:18:45.878253+0000","last_became_active":"2024-04-15T15:18:44.855203+0000","last_became_peered":"2024-04-15T15:18:44.855203+0000","last_unstale":"2024-04-15T15:18:45.878253+0000","last_undegraded":"2024-04-15T15:18:45.878253+0000","last_fullsized":"2024-04-15T15:18:45.878253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:34.796362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818209+0000","last_change":"2024-04-15T15:18:45.856661+0000","last_active":"2024-04-15T15:18:46.818209+0000","last_peered":"2024-04-15T15:18:46.818209+0000","last_clean":"2024-04-15T15:18:46.818209+0000","last_became_active":"2024-04-15T15:18:45.856443+0000","last_became_peered":"2024-04-15T15:18:45.856443+0000","last_unstale":"2024-04-15T15:18:46.818209+0000","last_undegraded":"2024-04-15T15:18:46.818209+0000","last_fullsized":"2024-04-15T15:18:46.818209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.566254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822757+0000","last_change":"2024-04-15T15:18:44.866401+0000","last_active":"2024-04-15T15:18:46.822757+0000","last_peered":"2024-04-15T15:18:46.822757+0000","last_clean":"2024-04-15T15:18:46.822757+0000","last_became_active":"2024-04-15T15:18:44.866099+0000","last_became_peered":"2024-04-15T15:18:44.866099+0000","last_unstale":"2024-04-15T15:18:46.822757+0000","last_undegraded":"2024-04-15T15:18:46.822757+0000","last_fullsized":"2024-04-15T15:18:46.822757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:15:03.621969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820130+0000","last_change":"2024-04-15T15:18:45.854183+0000","last_active":"2024-04-15T15:18:46.820130+0000","last_peered":"2024-04-15T15:18:46.820130+0000","last_clean":"2024-04-15T15:18:46.820130+0000","last_became_active":"2024-04-15T15:18:45.853850+0000","last_became_peered":"2024-04-15T15:18:45.853850+0000","last_unstale":"2024-04-15T15:18:46.820130+0000","last_undegraded":"2024-04-15T15:18:46.820130+0000","last_fullsized":"2024-04-15T15:18:46.820130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:32:58.852361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.876882+0000","last_change":"2024-04-15T15:18:44.857499+0000","last_active":"2024-04-15T15:18:45.876882+0000","last_peered":"2024-04-15T15:18:45.876882+0000","last_clean":"2024-04-15T15:18:45.876882+0000","last_became_active":"2024-04-15T15:18:44.857140+0000","last_became_peered":"2024-04-15T15:18:44.857140+0000","last_unstale":"2024-04-15T15:18:45.876882+0000","last_undegraded":"2024-04-15T15:18:45.876882+0000","last_fullsized":"2024-04-15T15:18:45.876882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.826627+0000","last_change":"2024-04-15T15:18:45.860607+0000","last_active":"2024-04-15T15:18:46.826627+0000","last_peered":"2024-04-15T15:18:46.826627+0000","last_clean":"2024-04-15T15:18:46.826627+0000","last_became_active":"2024-04-15T15:18:45.858730+0000","last_became_peered":"2024-04-15T15:18:45.858730+0000","last_unstale":"2024-04-15T15:18:46.826627+0000","last_undegraded":"2024-04-15T15:18:46.826627+0000","last_fullsized":"2024-04-15T15:18:46.826627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:10:16.717594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870214+0000","last_change":"2024-04-15T15:18:44.858372+0000","last_active":"2024-04-15T15:18:46.870214+0000","last_peered":"2024-04-15T15:18:46.870214+0000","last_clean":"2024-04-15T15:18:46.870214+0000","last_became_active":"2024-04-15T15:18:44.858083+0000","last_became_peered":"2024-04-15T15:18:44.858083+0000","last_unstale":"2024-04-15T15:18:46.870214+0000","last_undegraded":"2024-04-15T15:18:46.870214+0000","last_fullsized":"2024-04-15T15:18:46.870214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824118+0000","last_change":"2024-04-15T15:18:45.864746+0000","last_active":"2024-04-15T15:18:46.824118+0000","last_peered":"2024-04-15T15:18:46.824118+0000","last_clean":"2024-04-15T15:18:46.824118+0000","last_became_active":"2024-04-15T15:18:45.858273+0000","last_became_peered":"2024-04-15T15:18:45.858273+0000","last_unstale":"2024-04-15T15:18:46.824118+0000","last_undegraded":"2024-04-15T15:18:46.824118+0000","last_fullsized":"2024-04-15T15:18:46.824118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:36.969295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.835083+0000","last_change":"2024-04-15T15:18:44.864232+0000","last_active":"2024-04-15T15:18:45.835083+0000","last_peered":"2024-04-15T15:18:45.835083+0000","last_clean":"2024-04-15T15:18:45.835083+0000","last_became_active":"2024-04-15T15:18:44.863837+0000","last_became_peered":"2024-04-15T15:18:44.863837+0000","last_unstale":"2024-04-15T15:18:45.835083+0000","last_undegraded":"2024-04-15T15:18:45.835083+0000","last_fullsized":"2024-04-15T15:18:45.835083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818747+0000","last_change":"2024-04-15T15:18:45.860687+0000","last_active":"2024-04-15T15:18:46.818747+0000","last_peered":"2024-04-15T15:18:46.818747+0000","last_clean":"2024-04-15T15:18:46.818747+0000","last_became_active":"2024-04-15T15:18:45.860227+0000","last_became_peered":"2024-04-15T15:18:45.860227+0000","last_unstale":"2024-04-15T15:18:46.818747+0000","last_undegraded":"2024-04-15T15:18:46.818747+0000","last_fullsized":"2024-04-15T15:18:46.818747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:29:14.132066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.817987+0000","last_change":"2024-04-15T15:18:41.162887+0000","last_active":"2024-04-15T15:18:46.817987+0000","last_peered":"2024-04-15T15:18:46.817987+0000","last_clean":"2024-04-15T15:18:46.817987+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:18:46.817987+0000","last_undegraded":"2024-04-15T15:18:46.817987+0000","last_fullsized":"2024-04-15T15:18:46.817987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_clean_scrub_stamp":"2024-04-15T15:18:39.644354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:32:00.047414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821321+0000","last_change":"2024-04-15T15:18:44.858206+0000","last_active":"2024-04-15T15:18:46.821321+0000","last_peered":"2024-04-15T15:18:46.821321+0000","last_clean":"2024-04-15T15:18:46.821321+0000","last_became_active":"2024-04-15T15:18:44.857694+0000","last_became_peered":"2024-04-15T15:18:44.857694+0000","last_unstale":"2024-04-15T15:18:46.821321+0000","last_undegraded":"2024-04-15T15:18:46.821321+0000","last_fullsized":"2024-04-15T15:18:46.821321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.857312+0000","last_change":"2024-04-15T15:18:45.849224+0000","last_active":"2024-04-15T15:18:45.857312+0000","last_peered":"2024-04-15T15:18:45.857312+0000","last_clean":"2024-04-15T15:18:45.857312+0000","last_became_active":"2024-04-15T15:18:45.848961+0000","last_became_peered":"2024-04-15T15:18:45.848961+0000","last_unstale":"2024-04-15T15:18:45.857312+0000","last_undegraded":"2024-04-15T15:18:45.857312+0000","last_fullsized":"2024-04-15T15:18:45.857312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:18.082210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.817850+0000","last_change":"2024-04-15T15:18:44.859202+0000","last_active":"2024-04-15T15:18:46.817850+0000","last_peered":"2024-04-15T15:18:46.817850+0000","last_clean":"2024-04-15T15:18:46.817850+0000","last_became_active":"2024-04-15T15:18:44.858791+0000","last_became_peered":"2024-04-15T15:18:44.858791+0000","last_unstale":"2024-04-15T15:18:46.817850+0000","last_undegraded":"2024-04-15T15:18:46.817850+0000","last_fullsized":"2024-04-15T15:18:46.817850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.854716+0000","last_change":"2024-04-15T15:18:45.851981+0000","last_active":"2024-04-15T15:18:45.854716+0000","last_peered":"2024-04-15T15:18:45.854716+0000","last_clean":"2024-04-15T15:18:45.854716+0000","last_became_active":"2024-04-15T15:18:45.851653+0000","last_became_peered":"2024-04-15T15:18:45.851653+0000","last_unstale":"2024-04-15T15:18:45.854716+0000","last_undegraded":"2024-04-15T15:18:45.854716+0000","last_fullsized":"2024-04-15T15:18:45.854716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:51.536346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.833897+0000","last_change":"2024-04-15T15:18:44.849893+0000","last_active":"2024-04-15T15:18:45.833897+0000","last_peered":"2024-04-15T15:18:45.833897+0000","last_clean":"2024-04-15T15:18:45.833897+0000","last_became_active":"2024-04-15T15:18:44.848047+0000","last_became_peered":"2024-04-15T15:18:44.848047+0000","last_unstale":"2024-04-15T15:18:45.833897+0000","last_undegraded":"2024-04-15T15:18:45.833897+0000","last_fullsized":"2024-04-15T15:18:45.833897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869862+0000","last_change":"2024-04-15T15:18:45.844515+0000","last_active":"2024-04-15T15:18:46.869862+0000","last_peered":"2024-04-15T15:18:46.869862+0000","last_clean":"2024-04-15T15:18:46.869862+0000","last_became_active":"2024-04-15T15:18:45.844154+0000","last_became_peered":"2024-04-15T15:18:45.844154+0000","last_unstale":"2024-04-15T15:18:46.869862+0000","last_undegraded":"2024-04-15T15:18:46.869862+0000","last_fullsized":"2024-04-15T15:18:46.869862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823178+0000","last_change":"2024-04-15T15:18:44.862254+0000","last_active":"2024-04-15T15:18:46.823178+0000","last_peered":"2024-04-15T15:18:46.823178+0000","last_clean":"2024-04-15T15:18:46.823178+0000","last_became_active":"2024-04-15T15:18:44.861231+0000","last_became_peered":"2024-04-15T15:18:44.861231+0000","last_unstale":"2024-04-15T15:18:46.823178+0000","last_undegraded":"2024-04-15T15:18:46.823178+0000","last_fullsized":"2024-04-15T15:18:46.823178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821451+0000","last_change":"2024-04-15T15:18:45.847966+0000","last_active":"2024-04-15T15:18:46.821451+0000","last_peered":"2024-04-15T15:18:46.821451+0000","last_clean":"2024-04-15T15:18:46.821451+0000","last_became_active":"2024-04-15T15:18:45.847628+0000","last_became_peered":"2024-04-15T15:18:45.847628+0000","last_unstale":"2024-04-15T15:18:46.821451+0000","last_undegraded":"2024-04-15T15:18:46.821451+0000","last_fullsized":"2024-04-15T15:18:46.821451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:35:55.040215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870108+0000","last_change":"2024-04-15T15:18:44.866681+0000","last_active":"2024-04-15T15:18:46.870108+0000","last_peered":"2024-04-15T15:18:46.870108+0000","last_clean":"2024-04-15T15:18:46.870108+0000","last_became_active":"2024-04-15T15:18:44.866366+0000","last_became_peered":"2024-04-15T15:18:44.866366+0000","last_unstale":"2024-04-15T15:18:46.870108+0000","last_undegraded":"2024-04-15T15:18:46.870108+0000","last_fullsized":"2024-04-15T15:18:46.870108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824652+0000","last_change":"2024-04-15T15:18:45.858129+0000","last_active":"2024-04-15T15:18:46.824652+0000","last_peered":"2024-04-15T15:18:46.824652+0000","last_clean":"2024-04-15T15:18:46.824652+0000","last_became_active":"2024-04-15T15:18:45.857824+0000","last_became_peered":"2024-04-15T15:18:45.857824+0000","last_unstale":"2024-04-15T15:18:46.824652+0000","last_undegraded":"2024-04-15T15:18:46.824652+0000","last_fullsized":"2024-04-15T15:18:46.824652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:36:03.614062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820284+0000","last_change":"2024-04-15T15:18:44.866754+0000","last_active":"2024-04-15T15:18:46.820284+0000","last_peered":"2024-04-15T15:18:46.820284+0000","last_clean":"2024-04-15T15:18:46.820284+0000","last_became_active":"2024-04-15T15:18:44.866479+0000","last_became_peered":"2024-04-15T15:18:44.866479+0000","last_unstale":"2024-04-15T15:18:46.820284+0000","last_undegraded":"2024-04-15T15:18:46.820284+0000","last_fullsized":"2024-04-15T15:18:46.820284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:17:09.815187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870141+0000","last_change":"2024-04-15T15:18:45.844997+0000","last_active":"2024-04-15T15:18:46.870141+0000","last_peered":"2024-04-15T15:18:46.870141+0000","last_clean":"2024-04-15T15:18:46.870141+0000","last_became_active":"2024-04-15T15:18:45.844589+0000","last_became_peered":"2024-04-15T15:18:45.844589+0000","last_unstale":"2024-04-15T15:18:46.870141+0000","last_undegraded":"2024-04-15T15:18:46.870141+0000","last_fullsized":"2024-04-15T15:18:46.870141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.873141+0000","last_change":"2024-04-15T15:18:44.861663+0000","last_active":"2024-04-15T15:18:46.873141+0000","last_peered":"2024-04-15T15:18:46.873141+0000","last_clean":"2024-04-15T15:18:46.873141+0000","last_became_active":"2024-04-15T15:18:44.861347+0000","last_became_peered":"2024-04-15T15:18:44.861347+0000","last_unstale":"2024-04-15T15:18:46.873141+0000","last_undegraded":"2024-04-15T15:18:46.873141+0000","last_fullsized":"2024-04-15T15:18:46.873141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:54.111084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858205+0000","last_change":"2024-04-15T15:18:45.854665+0000","last_active":"2024-04-15T15:18:45.858205+0000","last_peered":"2024-04-15T15:18:45.858205+0000","last_clean":"2024-04-15T15:18:45.858205+0000","last_became_active":"2024-04-15T15:18:45.853958+0000","last_became_peered":"2024-04-15T15:18:45.853958+0000","last_unstale":"2024-04-15T15:18:45.858205+0000","last_undegraded":"2024-04-15T15:18:45.858205+0000","last_fullsized":"2024-04-15T15:18:45.858205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:57:54.115969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875345+0000","last_change":"2024-04-15T15:18:44.866078+0000","last_active":"2024-04-15T15:18:46.875345+0000","last_peered":"2024-04-15T15:18:46.875345+0000","last_clean":"2024-04-15T15:18:46.875345+0000","last_became_active":"2024-04-15T15:18:44.865730+0000","last_became_peered":"2024-04-15T15:18:44.865730+0000","last_unstale":"2024-04-15T15:18:46.875345+0000","last_undegraded":"2024-04-15T15:18:46.875345+0000","last_fullsized":"2024-04-15T15:18:46.875345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.844098+0000","last_change":"2024-04-15T15:18:45.841162+0000","last_active":"2024-04-15T15:18:45.844098+0000","last_peered":"2024-04-15T15:18:45.844098+0000","last_clean":"2024-04-15T15:18:45.844098+0000","last_became_active":"2024-04-15T15:18:45.840894+0000","last_became_peered":"2024-04-15T15:18:45.840894+0000","last_unstale":"2024-04-15T15:18:45.844098+0000","last_undegraded":"2024-04-15T15:18:45.844098+0000","last_fullsized":"2024-04-15T15:18:45.844098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:18.082210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819201+0000","last_change":"2024-04-15T15:18:44.862312+0000","last_active":"2024-04-15T15:18:46.819201+0000","last_peered":"2024-04-15T15:18:46.819201+0000","last_clean":"2024-04-15T15:18:46.819201+0000","last_became_active":"2024-04-15T15:18:44.862026+0000","last_became_peered":"2024-04-15T15:18:44.862026+0000","last_unstale":"2024-04-15T15:18:46.819201+0000","last_undegraded":"2024-04-15T15:18:46.819201+0000","last_fullsized":"2024-04-15T15:18:46.819201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822643+0000","last_change":"2024-04-15T15:18:45.847316+0000","last_active":"2024-04-15T15:18:46.822643+0000","last_peered":"2024-04-15T15:18:46.822643+0000","last_clean":"2024-04-15T15:18:46.822643+0000","last_became_active":"2024-04-15T15:18:45.846987+0000","last_became_peered":"2024-04-15T15:18:45.846987+0000","last_unstale":"2024-04-15T15:18:46.822643+0000","last_undegraded":"2024-04-15T15:18:46.822643+0000","last_fullsized":"2024-04-15T15:18:46.822643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:58:24.586889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869976+0000","last_change":"2024-04-15T15:18:44.859986+0000","last_active":"2024-04-15T15:18:46.869976+0000","last_peered":"2024-04-15T15:18:46.869976+0000","last_clean":"2024-04-15T15:18:46.869976+0000","last_became_active":"2024-04-15T15:18:44.859733+0000","last_became_peered":"2024-04-15T15:18:44.859733+0000","last_unstale":"2024-04-15T15:18:46.869976+0000","last_undegraded":"2024-04-15T15:18:46.869976+0000","last_fullsized":"2024-04-15T15:18:46.869976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:26.555787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.855272+0000","last_change":"2024-04-15T15:18:45.846102+0000","last_active":"2024-04-15T15:18:45.855272+0000","last_peered":"2024-04-15T15:18:45.855272+0000","last_clean":"2024-04-15T15:18:45.855272+0000","last_became_active":"2024-04-15T15:18:45.845715+0000","last_became_peered":"2024-04-15T15:18:45.845715+0000","last_unstale":"2024-04-15T15:18:45.855272+0000","last_undegraded":"2024-04-15T15:18:45.855272+0000","last_fullsized":"2024-04-15T15:18:45.855272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:50:45.496232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869992+0000","last_change":"2024-04-15T15:18:44.862018+0000","last_active":"2024-04-15T15:18:46.869992+0000","last_peered":"2024-04-15T15:18:46.869992+0000","last_clean":"2024-04-15T15:18:46.869992+0000","last_became_active":"2024-04-15T15:18:44.861663+0000","last_became_peered":"2024-04-15T15:18:44.861663+0000","last_unstale":"2024-04-15T15:18:46.869992+0000","last_undegraded":"2024-04-15T15:18:46.869992+0000","last_fullsized":"2024-04-15T15:18:46.869992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858232+0000","last_change":"2024-04-15T15:18:45.852079+0000","last_active":"2024-04-15T15:18:45.858232+0000","last_peered":"2024-04-15T15:18:45.858232+0000","last_clean":"2024-04-15T15:18:45.858232+0000","last_became_active":"2024-04-15T15:18:45.851281+0000","last_became_peered":"2024-04-15T15:18:45.851281+0000","last_unstale":"2024-04-15T15:18:45.858232+0000","last_undegraded":"2024-04-15T15:18:45.858232+0000","last_fullsized":"2024-04-15T15:18:45.858232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:55.246279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819899+0000","last_change":"2024-04-15T15:18:44.867447+0000","last_active":"2024-04-15T15:18:46.819899+0000","last_peered":"2024-04-15T15:18:46.819899+0000","last_clean":"2024-04-15T15:18:46.819899+0000","last_became_active":"2024-04-15T15:18:44.867171+0000","last_became_peered":"2024-04-15T15:18:44.867171+0000","last_unstale":"2024-04-15T15:18:46.819899+0000","last_undegraded":"2024-04-15T15:18:46.819899+0000","last_fullsized":"2024-04-15T15:18:46.819899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:26.555787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874593+0000","last_change":"2024-04-15T15:18:45.861329+0000","last_active":"2024-04-15T15:18:46.874593+0000","last_peered":"2024-04-15T15:18:46.874593+0000","last_clean":"2024-04-15T15:18:46.874593+0000","last_became_active":"2024-04-15T15:18:45.859564+0000","last_became_peered":"2024-04-15T15:18:45.859564+0000","last_unstale":"2024-04-15T15:18:46.874593+0000","last_undegraded":"2024-04-15T15:18:46.874593+0000","last_fullsized":"2024-04-15T15:18:46.874593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824729+0000","last_change":"2024-04-15T15:18:45.831940+0000","last_active":"2024-04-15T15:18:46.824729+0000","last_peered":"2024-04-15T15:18:46.824729+0000","last_clean":"2024-04-15T15:18:46.824729+0000","last_became_active":"2024-04-15T15:18:45.829915+0000","last_became_peered":"2024-04-15T15:18:45.829915+0000","last_unstale":"2024-04-15T15:18:46.824729+0000","last_undegraded":"2024-04-15T15:18:46.824729+0000","last_fullsized":"2024-04-15T15:18:46.824729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:06:25.954239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818726+0000","last_change":"2024-04-15T15:18:44.869358+0000","last_active":"2024-04-15T15:18:46.818726+0000","last_peered":"2024-04-15T15:18:46.818726+0000","last_clean":"2024-04-15T15:18:46.818726+0000","last_became_active":"2024-04-15T15:18:44.869040+0000","last_became_peered":"2024-04-15T15:18:44.869040+0000","last_unstale":"2024-04-15T15:18:46.818726+0000","last_undegraded":"2024-04-15T15:18:46.818726+0000","last_fullsized":"2024-04-15T15:18:46.818726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:54.111084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823654+0000","last_change":"2024-04-15T15:18:45.851111+0000","last_active":"2024-04-15T15:18:46.823654+0000","last_peered":"2024-04-15T15:18:46.823654+0000","last_clean":"2024-04-15T15:18:46.823654+0000","last_became_active":"2024-04-15T15:18:45.850824+0000","last_became_peered":"2024-04-15T15:18:45.850824+0000","last_unstale":"2024-04-15T15:18:46.823654+0000","last_undegraded":"2024-04-15T15:18:46.823654+0000","last_fullsized":"2024-04-15T15:18:46.823654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:53.984837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818964+0000","last_change":"2024-04-15T15:18:44.864267+0000","last_active":"2024-04-15T15:18:46.818964+0000","last_peered":"2024-04-15T15:18:46.818964+0000","last_clean":"2024-04-15T15:18:46.818964+0000","last_became_active":"2024-04-15T15:18:44.863962+0000","last_became_peered":"2024-04-15T15:18:44.863962+0000","last_unstale":"2024-04-15T15:18:46.818964+0000","last_undegraded":"2024-04-15T15:18:46.818964+0000","last_fullsized":"2024-04-15T15:18:46.818964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:22:53.739928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.852693+0000","last_change":"2024-04-15T15:18:45.845359+0000","last_active":"2024-04-15T15:18:45.852693+0000","last_peered":"2024-04-15T15:18:45.852693+0000","last_clean":"2024-04-15T15:18:45.852693+0000","last_became_active":"2024-04-15T15:18:45.845087+0000","last_became_peered":"2024-04-15T15:18:45.845087+0000","last_unstale":"2024-04-15T15:18:45.852693+0000","last_undegraded":"2024-04-15T15:18:45.852693+0000","last_fullsized":"2024-04-15T15:18:45.852693+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:20.202506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.875540+0000","last_change":"2024-04-15T15:18:44.864497+0000","last_active":"2024-04-15T15:18:45.875540+0000","last_peered":"2024-04-15T15:18:45.875540+0000","last_clean":"2024-04-15T15:18:45.875540+0000","last_became_active":"2024-04-15T15:18:44.864122+0000","last_became_peered":"2024-04-15T15:18:44.864122+0000","last_unstale":"2024-04-15T15:18:45.875540+0000","last_undegraded":"2024-04-15T15:18:45.875540+0000","last_fullsized":"2024-04-15T15:18:45.875540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.863115+0000","last_change":"2024-04-15T15:18:45.861978+0000","last_active":"2024-04-15T15:18:45.863115+0000","last_peered":"2024-04-15T15:18:45.863115+0000","last_clean":"2024-04-15T15:18:45.863115+0000","last_became_active":"2024-04-15T15:18:45.861597+0000","last_became_peered":"2024-04-15T15:18:45.861597+0000","last_unstale":"2024-04-15T15:18:45.863115+0000","last_undegraded":"2024-04-15T15:18:45.863115+0000","last_fullsized":"2024-04-15T15:18:45.863115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:16:52.507783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.879016+0000","last_change":"2024-04-15T15:18:44.864325+0000","last_active":"2024-04-15T15:18:45.879016+0000","last_peered":"2024-04-15T15:18:45.879016+0000","last_clean":"2024-04-15T15:18:45.879016+0000","last_became_active":"2024-04-15T15:18:44.863985+0000","last_became_peered":"2024-04-15T15:18:44.863985+0000","last_unstale":"2024-04-15T15:18:45.879016+0000","last_undegraded":"2024-04-15T15:18:45.879016+0000","last_fullsized":"2024-04-15T15:18:45.879016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":811008,"data_stored":663558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":72734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":74220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":72608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":79068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":819200,"data_stored":663614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:18:50.377 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:18:50.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.634+0000 7f6c75400640 1 Processor -- start 2024-04-15T15:18:50.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.634+0000 7f6c75400640 1 -- start start 2024-04-15T15:18:50.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.636+0000 7f6c75400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6c7015ebb0 0x7f6c70163060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.636+0000 7f6c75400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b3c70 0x7f6c700d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.636+0000 7f6c75400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c701635a0 con 0x7f6c700d1af0 2024-04-15T15:18:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.636+0000 7f6c75400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f6c700d1f60 con 0x7f6c7015ebb0 2024-04-15T15:18:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.636+0000 7f6c75400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7015ef90 con 0x7f6c700b3c70 2024-04-15T15:18:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.637+0000 7f6c6f400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b3c70 0x7f6c700d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.637+0000 7f6c6f400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b3c70 0x7f6c700d15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:45900/0 (socket says 172.21.15.49:45900) 2024-04-15T15:18:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.637+0000 7f6c6f400640 1 -- 172.21.15.49:0/3778448382 learned_addr learned my addr 172.21.15.49:0/3778448382 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.637+0000 7f6c6fe00640 1 --2- 172.21.15.49:0/3778448382 >> v2:172.21.15.107:3300/0 conn(0x7f6c7015ebb0 0x7f6c70163060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.637+0000 7f6c6f400640 1 -- 172.21.15.49:0/3778448382 >> v2:172.21.15.107:3300/0 conn(0x7f6c7015ebb0 msgr2=0x7f6c70163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.637+0000 7f6c6f400640 1 --2- 172.21.15.49:0/3778448382 >> v2:172.21.15.107:3300/0 conn(0x7f6c7015ebb0 0x7f6c70163060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.637+0000 7f6c6f400640 1 -- 172.21.15.49:0/3778448382 >> v1:172.21.15.107:6789/0 conn(0x7f6c700d1af0 legacy=0x7f6c700b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.637+0000 7f6c6ea00640 1 -- 172.21.15.49:0/3778448382 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2605631947 0 0) 0x7f6c701635a0 con 0x7f6c700d1af0 2024-04-15T15:18:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.638+0000 7f6c6f400640 1 -- 172.21.15.49:0/3778448382 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c700cf090 con 0x7f6c700b3c70 2024-04-15T15:18:50.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.639+0000 7f6c6f400640 1 --2- 172.21.15.49:0/3778448382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b3c70 0x7f6c700d15b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009870 tx=0x7f6c5c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=970b41076b8a0925 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:50.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.639+0000 7f6c6ea00640 1 -- 172.21.15.49:0/3778448382 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c034070 con 0x7f6c700b3c70 2024-04-15T15:18:50.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.639+0000 7f6c6ea00640 1 -- 172.21.15.49:0/3778448382 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c5c030840 con 0x7f6c700b3c70 2024-04-15T15:18:50.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.639+0000 7f6c6ea00640 1 -- 172.21.15.49:0/3778448382 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c030c10 con 0x7f6c700b3c70 2024-04-15T15:18:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.640+0000 7f6c75400640 1 -- 172.21.15.49:0/3778448382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b3c70 msgr2=0x7f6c700d15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.640+0000 7f6c75400640 1 --2- 172.21.15.49:0/3778448382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b3c70 0x7f6c700d15b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009870 tx=0x7f6c5c02b980 comp rx=0 tx=0).stop 2024-04-15T15:18:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.640+0000 7f6c75400640 1 -- 172.21.15.49:0/3778448382 shutdown_connections 2024-04-15T15:18:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.640+0000 7f6c75400640 1 --2- 172.21.15.49:0/3778448382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b3c70 0x7f6c700d15b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.640+0000 7f6c75400640 1 --2- 172.21.15.49:0/3778448382 >> v2:172.21.15.107:3300/0 conn(0x7f6c7015ebb0 0x7f6c70163060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.640+0000 7f6c75400640 1 -- 172.21.15.49:0/3778448382 >> 172.21.15.49:0/3778448382 conn(0x7f6c7006d040 msgr2=0x7f6c7006d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.640+0000 7f6c75400640 1 -- 172.21.15.49:0/3778448382 shutdown_connections 2024-04-15T15:18:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.640+0000 7f6c75400640 1 -- 172.21.15.49:0/3778448382 wait complete. 2024-04-15T15:18:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.642+0000 7f6c75400640 1 Processor -- start 2024-04-15T15:18:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.642+0000 7f6c75400640 1 -- start start 2024-04-15T15:18:50.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.643+0000 7f6c75400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700d1af0 0x7f6c70155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.643+0000 7f6c75400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c7015ebb0 0x7f6c700b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.643+0000 7f6c75400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b5050 0x7f6c700b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.643+0000 7f6c75400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c700cf930 con 0x7f6c700d1af0 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.643+0000 7f6c75400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c700cf650 con 0x7f6c7015ebb0 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.643+0000 7f6c75400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6c700cfc10 con 0x7f6c700b5050 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.643+0000 7f6c6fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700d1af0 0x7f6c70155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.643+0000 7f6c6fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700d1af0 0x7f6c70155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58318/0 (socket says 172.21.15.49:58318) 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.643+0000 7f6c6fe00640 1 -- 172.21.15.49:0/1121756910 learned_addr learned my addr 172.21.15.49:0/1121756910 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.644+0000 7f6c74a00640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b5050 0x7f6c700b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.644+0000 7f6c6f400640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c7015ebb0 0x7f6c700b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.644+0000 7f6c6fe00640 1 -- 172.21.15.49:0/1121756910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b5050 msgr2=0x7f6c700b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.644+0000 7f6c6fe00640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b5050 0x7f6c700b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.644+0000 7f6c6fe00640 1 -- 172.21.15.49:0/1121756910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c7015ebb0 msgr2=0x7f6c700b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.644+0000 7f6c6fe00640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c7015ebb0 0x7f6c700b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.644+0000 7f6c6fe00640 1 -- 172.21.15.49:0/1121756910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c700c8340 con 0x7f6c700d1af0 2024-04-15T15:18:50.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.644+0000 7f6c6fe00640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700d1af0 0x7f6c70155dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000e870 tx=0x7f6c6000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:50.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.645+0000 7f6c67e00640 1 -- 172.21.15.49:0/1121756910 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60017070 con 0x7f6c700d1af0 2024-04-15T15:18:50.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.645+0000 7f6c67e00640 1 -- 172.21.15.49:0/1121756910 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c6000c940 con 0x7f6c700d1af0 2024-04-15T15:18:50.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.645+0000 7f6c67e00640 1 -- 172.21.15.49:0/1121756910 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000cd30 con 0x7f6c700d1af0 2024-04-15T15:18:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.645+0000 7f6c75400640 1 -- 172.21.15.49:0/1121756910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c700c8620 con 0x7f6c700d1af0 2024-04-15T15:18:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.645+0000 7f6c75400640 1 -- 172.21.15.49:0/1121756910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c700c8be0 con 0x7f6c700d1af0 2024-04-15T15:18:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.646+0000 7f6c75400640 1 -- 172.21.15.49:0/1121756910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c700755c0 con 0x7f6c700d1af0 2024-04-15T15:18:50.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.647+0000 7f6c67e00640 1 -- 172.21.15.49:0/1121756910 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c6001a070 con 0x7f6c700d1af0 2024-04-15T15:18:50.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.648+0000 7f6c67e00640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c5006c940 0x7f6c5006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:50.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.648+0000 7f6c67e00640 1 -- 172.21.15.49:0/1121756910 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6c6008bcc0 con 0x7f6c700d1af0 2024-04-15T15:18:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.648+0000 7f6c6f400640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c5006c940 0x7f6c5006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:50.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.649+0000 7f6c6f400640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c5006c940 0x7f6c5006ee00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c036000 tx=0x7f6c5c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:50.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.653+0000 7f6c67e00640 1 -- 172.21.15.49:0/1121756910 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c6005a1d0 con 0x7f6c700d1af0 2024-04-15T15:18:50.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.854+0000 7f6c75400640 1 -- 172.21.15.49:0/1121756910 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c700d2540 con 0x7f6c5006c940 2024-04-15T15:18:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.869+0000 7f6c67e00640 1 -- 172.21.15.49:0/1121756910 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f6c700d2540 con 0x7f6c5006c940 2024-04-15T15:18:50.874 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:50.875 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:18:50.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.873+0000 7f6c75400640 1 -- 172.21.15.49:0/1121756910 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c5006c940 msgr2=0x7f6c5006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.873+0000 7f6c75400640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c5006c940 0x7f6c5006ee00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c036000 tx=0x7f6c5c008030 comp rx=0 tx=0).stop 2024-04-15T15:18:50.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.873+0000 7f6c75400640 1 -- 172.21.15.49:0/1121756910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700d1af0 msgr2=0x7f6c70155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:50.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.873+0000 7f6c75400640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700d1af0 0x7f6c70155dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000e870 tx=0x7f6c6000ed30 comp rx=0 tx=0).stop 2024-04-15T15:18:50.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.874+0000 7f6c75400640 1 -- 172.21.15.49:0/1121756910 shutdown_connections 2024-04-15T15:18:50.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.874+0000 7f6c75400640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c5006c940 0x7f6c5006ee00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.874+0000 7f6c75400640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c700b5050 0x7f6c700b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.874+0000 7f6c75400640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c7015ebb0 0x7f6c700b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.874+0000 7f6c75400640 1 --2- 172.21.15.49:0/1121756910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700d1af0 0x7f6c70155dc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:50.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.874+0000 7f6c75400640 1 -- 172.21.15.49:0/1121756910 >> 172.21.15.49:0/1121756910 conn(0x7f6c7006d040 msgr2=0x7f6c7015c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:50.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.874+0000 7f6c75400640 1 -- 172.21.15.49:0/1121756910 shutdown_connections 2024-04-15T15:18:50.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:50.874+0000 7f6c75400640 1 -- 172.21.15.49:0/1121756910 wait complete. 2024-04-15T15:18:50.895 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-15T15:18:50.772283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236432,"kb_used_data":2900,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738288,"statfs":{"total":773094113280,"available":772852006912,"internally_reserved":0,"allocated":2969600,"data_stored":1772720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":62,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002126"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.825719+0000","last_change":"2024-04-15T15:18:45.850244+0000","last_active":"2024-04-15T15:18:46.825719+0000","last_peered":"2024-04-15T15:18:46.825719+0000","last_clean":"2024-04-15T15:18:46.825719+0000","last_became_active":"2024-04-15T15:18:45.849857+0000","last_became_peered":"2024-04-15T15:18:45.849857+0000","last_unstale":"2024-04-15T15:18:46.825719+0000","last_undegraded":"2024-04-15T15:18:46.825719+0000","last_fullsized":"2024-04-15T15:18:46.825719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:08:04.293347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870331+0000","last_change":"2024-04-15T15:18:45.843425+0000","last_active":"2024-04-15T15:18:46.870331+0000","last_peered":"2024-04-15T15:18:46.870331+0000","last_clean":"2024-04-15T15:18:46.870331+0000","last_became_active":"2024-04-15T15:18:45.843052+0000","last_became_peered":"2024-04-15T15:18:45.843052+0000","last_unstale":"2024-04-15T15:18:46.870331+0000","last_undegraded":"2024-04-15T15:18:46.870331+0000","last_fullsized":"2024-04-15T15:18:46.870331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.860635+0000","last_change":"2024-04-15T15:18:45.853920+0000","last_active":"2024-04-15T15:18:45.860635+0000","last_peered":"2024-04-15T15:18:45.860635+0000","last_clean":"2024-04-15T15:18:45.860635+0000","last_became_active":"2024-04-15T15:18:45.853659+0000","last_became_peered":"2024-04-15T15:18:45.853659+0000","last_unstale":"2024-04-15T15:18:45.860635+0000","last_undegraded":"2024-04-15T15:18:45.860635+0000","last_fullsized":"2024-04-15T15:18:45.860635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:27.690982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823670+0000","last_change":"2024-04-15T15:18:45.861695+0000","last_active":"2024-04-15T15:18:46.823670+0000","last_peered":"2024-04-15T15:18:46.823670+0000","last_clean":"2024-04-15T15:18:46.823670+0000","last_became_active":"2024-04-15T15:18:45.858560+0000","last_became_peered":"2024-04-15T15:18:45.858560+0000","last_unstale":"2024-04-15T15:18:46.823670+0000","last_undegraded":"2024-04-15T15:18:46.823670+0000","last_fullsized":"2024-04-15T15:18:46.823670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:29:14.132066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820061+0000","last_change":"2024-04-15T15:18:45.849530+0000","last_active":"2024-04-15T15:18:46.820061+0000","last_peered":"2024-04-15T15:18:46.820061+0000","last_clean":"2024-04-15T15:18:46.820061+0000","last_became_active":"2024-04-15T15:18:45.849255+0000","last_became_peered":"2024-04-15T15:18:45.849255+0000","last_unstale":"2024-04-15T15:18:46.820061+0000","last_undegraded":"2024-04-15T15:18:46.820061+0000","last_fullsized":"2024-04-15T15:18:46.820061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:36:03.614062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822193+0000","last_change":"2024-04-15T15:18:45.842283+0000","last_active":"2024-04-15T15:18:46.822193+0000","last_peered":"2024-04-15T15:18:46.822193+0000","last_clean":"2024-04-15T15:18:46.822193+0000","last_became_active":"2024-04-15T15:18:45.842004+0000","last_became_peered":"2024-04-15T15:18:45.842004+0000","last_unstale":"2024-04-15T15:18:46.822193+0000","last_undegraded":"2024-04-15T15:18:46.822193+0000","last_fullsized":"2024-04-15T15:18:46.822193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:08:29.977812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818174+0000","last_change":"2024-04-15T15:18:45.856124+0000","last_active":"2024-04-15T15:18:46.818174+0000","last_peered":"2024-04-15T15:18:46.818174+0000","last_clean":"2024-04-15T15:18:46.818174+0000","last_became_active":"2024-04-15T15:18:45.849034+0000","last_became_peered":"2024-04-15T15:18:45.849034+0000","last_unstale":"2024-04-15T15:18:46.818174+0000","last_undegraded":"2024-04-15T15:18:46.818174+0000","last_fullsized":"2024-04-15T15:18:46.818174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:49:43.342170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.855348+0000","last_change":"2024-04-15T15:18:45.848306+0000","last_active":"2024-04-15T15:18:45.855348+0000","last_peered":"2024-04-15T15:18:45.855348+0000","last_clean":"2024-04-15T15:18:45.855348+0000","last_became_active":"2024-04-15T15:18:45.848033+0000","last_became_peered":"2024-04-15T15:18:45.848033+0000","last_unstale":"2024-04-15T15:18:45.855348+0000","last_undegraded":"2024-04-15T15:18:45.855348+0000","last_fullsized":"2024-04-15T15:18:45.855348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:35.931556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.826189+0000","last_change":"2024-04-15T15:18:45.862452+0000","last_active":"2024-04-15T15:18:46.826189+0000","last_peered":"2024-04-15T15:18:46.826189+0000","last_clean":"2024-04-15T15:18:46.826189+0000","last_became_active":"2024-04-15T15:18:45.862157+0000","last_became_peered":"2024-04-15T15:18:45.862157+0000","last_unstale":"2024-04-15T15:18:46.826189+0000","last_undegraded":"2024-04-15T15:18:46.826189+0000","last_fullsized":"2024-04-15T15:18:46.826189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:14:17.068828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875290+0000","last_change":"2024-04-15T15:18:45.855709+0000","last_active":"2024-04-15T15:18:46.875290+0000","last_peered":"2024-04-15T15:18:46.875290+0000","last_clean":"2024-04-15T15:18:46.875290+0000","last_became_active":"2024-04-15T15:18:45.855243+0000","last_became_peered":"2024-04-15T15:18:45.855243+0000","last_unstale":"2024-04-15T15:18:46.875290+0000","last_undegraded":"2024-04-15T15:18:46.875290+0000","last_fullsized":"2024-04-15T15:18:46.875290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874937+0000","last_change":"2024-04-15T15:18:45.859684+0000","last_active":"2024-04-15T15:18:46.874937+0000","last_peered":"2024-04-15T15:18:46.874937+0000","last_clean":"2024-04-15T15:18:46.874937+0000","last_became_active":"2024-04-15T15:18:45.856683+0000","last_became_peered":"2024-04-15T15:18:45.856683+0000","last_unstale":"2024-04-15T15:18:46.874937+0000","last_undegraded":"2024-04-15T15:18:46.874937+0000","last_fullsized":"2024-04-15T15:18:46.874937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822415+0000","last_change":"2024-04-15T15:18:45.849769+0000","last_active":"2024-04-15T15:18:46.822415+0000","last_peered":"2024-04-15T15:18:46.822415+0000","last_clean":"2024-04-15T15:18:46.822415+0000","last_became_active":"2024-04-15T15:18:45.849462+0000","last_became_peered":"2024-04-15T15:18:45.849462+0000","last_unstale":"2024-04-15T15:18:46.822415+0000","last_undegraded":"2024-04-15T15:18:46.822415+0000","last_fullsized":"2024-04-15T15:18:46.822415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:35:53.211120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858253+0000","last_change":"2024-04-15T15:18:45.852551+0000","last_active":"2024-04-15T15:18:45.858253+0000","last_peered":"2024-04-15T15:18:45.858253+0000","last_clean":"2024-04-15T15:18:45.858253+0000","last_became_active":"2024-04-15T15:18:45.851658+0000","last_became_peered":"2024-04-15T15:18:45.851658+0000","last_unstale":"2024-04-15T15:18:45.858253+0000","last_undegraded":"2024-04-15T15:18:45.858253+0000","last_fullsized":"2024-04-15T15:18:45.858253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:27.690982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.849914+0000","last_change":"2024-04-15T15:18:45.842127+0000","last_active":"2024-04-15T15:18:45.849914+0000","last_peered":"2024-04-15T15:18:45.849914+0000","last_clean":"2024-04-15T15:18:45.849914+0000","last_became_active":"2024-04-15T15:18:45.841662+0000","last_became_peered":"2024-04-15T15:18:45.841662+0000","last_unstale":"2024-04-15T15:18:45.849914+0000","last_undegraded":"2024-04-15T15:18:45.849914+0000","last_fullsized":"2024-04-15T15:18:45.849914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:22.780040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869891+0000","last_change":"2024-04-15T15:18:45.832097+0000","last_active":"2024-04-15T15:18:46.869891+0000","last_peered":"2024-04-15T15:18:46.869891+0000","last_clean":"2024-04-15T15:18:46.869891+0000","last_became_active":"2024-04-15T15:18:45.831411+0000","last_became_peered":"2024-04-15T15:18:45.831411+0000","last_unstale":"2024-04-15T15:18:46.869891+0000","last_undegraded":"2024-04-15T15:18:46.869891+0000","last_fullsized":"2024-04-15T15:18:46.869891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822037+0000","last_change":"2024-04-15T15:18:45.846022+0000","last_active":"2024-04-15T15:18:46.822037+0000","last_peered":"2024-04-15T15:18:46.822037+0000","last_clean":"2024-04-15T15:18:46.822037+0000","last_became_active":"2024-04-15T15:18:45.844025+0000","last_became_peered":"2024-04-15T15:18:45.844025+0000","last_unstale":"2024-04-15T15:18:46.822037+0000","last_undegraded":"2024-04-15T15:18:46.822037+0000","last_fullsized":"2024-04-15T15:18:46.822037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.825219+0000","last_change":"2024-04-15T15:18:45.849397+0000","last_active":"2024-04-15T15:18:46.825219+0000","last_peered":"2024-04-15T15:18:46.825219+0000","last_clean":"2024-04-15T15:18:46.825219+0000","last_became_active":"2024-04-15T15:18:45.849114+0000","last_became_peered":"2024-04-15T15:18:45.849114+0000","last_unstale":"2024-04-15T15:18:46.825219+0000","last_undegraded":"2024-04-15T15:18:46.825219+0000","last_fullsized":"2024-04-15T15:18:46.825219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:08:04.293347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.850585+0000","last_change":"2024-04-15T15:18:45.848118+0000","last_active":"2024-04-15T15:18:45.850585+0000","last_peered":"2024-04-15T15:18:45.850585+0000","last_clean":"2024-04-15T15:18:45.850585+0000","last_became_active":"2024-04-15T15:18:45.847644+0000","last_became_peered":"2024-04-15T15:18:45.847644+0000","last_unstale":"2024-04-15T15:18:45.850585+0000","last_undegraded":"2024-04-15T15:18:45.850585+0000","last_fullsized":"2024-04-15T15:18:45.850585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:51.536346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820072+0000","last_change":"2024-04-15T15:18:45.860524+0000","last_active":"2024-04-15T15:18:46.820072+0000","last_peered":"2024-04-15T15:18:46.820072+0000","last_clean":"2024-04-15T15:18:46.820072+0000","last_became_active":"2024-04-15T15:18:45.852020+0000","last_became_peered":"2024-04-15T15:18:45.852020+0000","last_unstale":"2024-04-15T15:18:46.820072+0000","last_undegraded":"2024-04-15T15:18:46.820072+0000","last_fullsized":"2024-04-15T15:18:46.820072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:06:25.954239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874881+0000","last_change":"2024-04-15T15:18:45.852135+0000","last_active":"2024-04-15T15:18:46.874881+0000","last_peered":"2024-04-15T15:18:46.874881+0000","last_clean":"2024-04-15T15:18:46.874881+0000","last_became_active":"2024-04-15T15:18:45.851841+0000","last_became_peered":"2024-04-15T15:18:45.851841+0000","last_unstale":"2024-04-15T15:18:46.874881+0000","last_undegraded":"2024-04-15T15:18:46.874881+0000","last_fullsized":"2024-04-15T15:18:46.874881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.861379+0000","last_change":"2024-04-15T15:18:45.859189+0000","last_active":"2024-04-15T15:18:45.861379+0000","last_peered":"2024-04-15T15:18:45.861379+0000","last_clean":"2024-04-15T15:18:45.861379+0000","last_became_active":"2024-04-15T15:18:45.857117+0000","last_became_peered":"2024-04-15T15:18:45.857117+0000","last_unstale":"2024-04-15T15:18:45.861379+0000","last_undegraded":"2024-04-15T15:18:45.861379+0000","last_fullsized":"2024-04-15T15:18:45.861379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:57:54.115969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822607+0000","last_change":"2024-04-15T15:18:45.840913+0000","last_active":"2024-04-15T15:18:46.822607+0000","last_peered":"2024-04-15T15:18:46.822607+0000","last_clean":"2024-04-15T15:18:46.822607+0000","last_became_active":"2024-04-15T15:18:45.840257+0000","last_became_peered":"2024-04-15T15:18:45.840257+0000","last_unstale":"2024-04-15T15:18:46.822607+0000","last_undegraded":"2024-04-15T15:18:46.822607+0000","last_fullsized":"2024-04-15T15:18:46.822607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:49:43.342170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.857965+0000","last_change":"2024-04-15T15:18:45.851064+0000","last_active":"2024-04-15T15:18:45.857965+0000","last_peered":"2024-04-15T15:18:45.857965+0000","last_clean":"2024-04-15T15:18:45.857965+0000","last_became_active":"2024-04-15T15:18:45.850725+0000","last_became_peered":"2024-04-15T15:18:45.850725+0000","last_unstale":"2024-04-15T15:18:45.857965+0000","last_undegraded":"2024-04-15T15:18:45.857965+0000","last_fullsized":"2024-04-15T15:18:45.857965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:22.780040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820869+0000","last_change":"2024-04-15T15:18:45.838476+0000","last_active":"2024-04-15T15:18:46.820869+0000","last_peered":"2024-04-15T15:18:46.820869+0000","last_clean":"2024-04-15T15:18:46.820869+0000","last_became_active":"2024-04-15T15:18:45.838160+0000","last_became_peered":"2024-04-15T15:18:45.838160+0000","last_unstale":"2024-04-15T15:18:46.820869+0000","last_undegraded":"2024-04-15T15:18:46.820869+0000","last_fullsized":"2024-04-15T15:18:46.820869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858227+0000","last_change":"2024-04-15T15:18:45.855537+0000","last_active":"2024-04-15T15:18:45.858227+0000","last_peered":"2024-04-15T15:18:45.858227+0000","last_clean":"2024-04-15T15:18:45.858227+0000","last_became_active":"2024-04-15T15:18:45.855144+0000","last_became_peered":"2024-04-15T15:18:45.855144+0000","last_unstale":"2024-04-15T15:18:45.858227+0000","last_undegraded":"2024-04-15T15:18:45.858227+0000","last_fullsized":"2024-04-15T15:18:45.858227+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:18.727426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875245+0000","last_change":"2024-04-15T15:18:45.848420+0000","last_active":"2024-04-15T15:18:46.875245+0000","last_peered":"2024-04-15T15:18:46.875245+0000","last_clean":"2024-04-15T15:18:46.875245+0000","last_became_active":"2024-04-15T15:18:45.848080+0000","last_became_peered":"2024-04-15T15:18:45.848080+0000","last_unstale":"2024-04-15T15:18:46.875245+0000","last_undegraded":"2024-04-15T15:18:46.875245+0000","last_fullsized":"2024-04-15T15:18:46.875245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869996+0000","last_change":"2024-04-15T15:18:45.858358+0000","last_active":"2024-04-15T15:18:46.869996+0000","last_peered":"2024-04-15T15:18:46.869996+0000","last_clean":"2024-04-15T15:18:46.869996+0000","last_became_active":"2024-04-15T15:18:45.856289+0000","last_became_peered":"2024-04-15T15:18:45.856289+0000","last_unstale":"2024-04-15T15:18:46.869996+0000","last_undegraded":"2024-04-15T15:18:46.869996+0000","last_fullsized":"2024-04-15T15:18:46.869996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858144+0000","last_change":"2024-04-15T15:18:45.854349+0000","last_active":"2024-04-15T15:18:45.858144+0000","last_peered":"2024-04-15T15:18:45.858144+0000","last_clean":"2024-04-15T15:18:45.858144+0000","last_became_active":"2024-04-15T15:18:45.853978+0000","last_became_peered":"2024-04-15T15:18:45.853978+0000","last_unstale":"2024-04-15T15:18:45.858144+0000","last_undegraded":"2024-04-15T15:18:45.858144+0000","last_fullsized":"2024-04-15T15:18:45.858144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:17:10.950382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.861076+0000","last_change":"2024-04-15T15:18:45.858661+0000","last_active":"2024-04-15T15:18:45.861076+0000","last_peered":"2024-04-15T15:18:45.861076+0000","last_clean":"2024-04-15T15:18:45.861076+0000","last_became_active":"2024-04-15T15:18:45.858244+0000","last_became_peered":"2024-04-15T15:18:45.858244+0000","last_unstale":"2024-04-15T15:18:45.861076+0000","last_undegraded":"2024-04-15T15:18:45.861076+0000","last_fullsized":"2024-04-15T15:18:45.861076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:17:10.950382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823029+0000","last_change":"2024-04-15T15:18:45.840590+0000","last_active":"2024-04-15T15:18:46.823029+0000","last_peered":"2024-04-15T15:18:46.823029+0000","last_clean":"2024-04-15T15:18:46.823029+0000","last_became_active":"2024-04-15T15:18:45.839849+0000","last_became_peered":"2024-04-15T15:18:45.839849+0000","last_unstale":"2024-04-15T15:18:46.823029+0000","last_undegraded":"2024-04-15T15:18:46.823029+0000","last_fullsized":"2024-04-15T15:18:46.823029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.566254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.850135+0000","last_change":"2024-04-15T15:18:45.847940+0000","last_active":"2024-04-15T15:18:45.850135+0000","last_peered":"2024-04-15T15:18:45.850135+0000","last_clean":"2024-04-15T15:18:45.850135+0000","last_became_active":"2024-04-15T15:18:45.846916+0000","last_became_peered":"2024-04-15T15:18:45.846916+0000","last_unstale":"2024-04-15T15:18:45.850135+0000","last_undegraded":"2024-04-15T15:18:45.850135+0000","last_fullsized":"2024-04-15T15:18:45.850135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:20.202506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874483+0000","last_change":"2024-04-15T15:18:45.846327+0000","last_active":"2024-04-15T15:18:46.874483+0000","last_peered":"2024-04-15T15:18:46.874483+0000","last_clean":"2024-04-15T15:18:46.874483+0000","last_became_active":"2024-04-15T15:18:45.846010+0000","last_became_peered":"2024-04-15T15:18:45.846010+0000","last_unstale":"2024-04-15T15:18:46.874483+0000","last_undegraded":"2024-04-15T15:18:46.874483+0000","last_fullsized":"2024-04-15T15:18:46.874483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820924+0000","last_change":"2024-04-15T15:18:45.846764+0000","last_active":"2024-04-15T15:18:46.820924+0000","last_peered":"2024-04-15T15:18:46.820924+0000","last_clean":"2024-04-15T15:18:46.820924+0000","last_became_active":"2024-04-15T15:18:45.842870+0000","last_became_peered":"2024-04-15T15:18:45.842870+0000","last_unstale":"2024-04-15T15:18:46.820924+0000","last_undegraded":"2024-04-15T15:18:46.820924+0000","last_fullsized":"2024-04-15T15:18:46.820924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:38:23.700029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.875822+0000","last_change":"2024-04-15T15:18:44.864457+0000","last_active":"2024-04-15T15:18:45.875822+0000","last_peered":"2024-04-15T15:18:45.875822+0000","last_clean":"2024-04-15T15:18:45.875822+0000","last_became_active":"2024-04-15T15:18:44.863999+0000","last_became_peered":"2024-04-15T15:18:44.863999+0000","last_unstale":"2024-04-15T15:18:45.875822+0000","last_undegraded":"2024-04-15T15:18:45.875822+0000","last_fullsized":"2024-04-15T15:18:45.875822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.861065+0000","last_change":"2024-04-15T15:18:45.855613+0000","last_active":"2024-04-15T15:18:45.861065+0000","last_peered":"2024-04-15T15:18:45.861065+0000","last_clean":"2024-04-15T15:18:45.861065+0000","last_became_active":"2024-04-15T15:18:45.855327+0000","last_became_peered":"2024-04-15T15:18:45.855327+0000","last_unstale":"2024-04-15T15:18:45.861065+0000","last_undegraded":"2024-04-15T15:18:45.861065+0000","last_fullsized":"2024-04-15T15:18:45.861065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:22:54.875123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874502+0000","last_change":"2024-04-15T15:18:44.864183+0000","last_active":"2024-04-15T15:18:46.874502+0000","last_peered":"2024-04-15T15:18:46.874502+0000","last_clean":"2024-04-15T15:18:46.874502+0000","last_became_active":"2024-04-15T15:18:44.861740+0000","last_became_peered":"2024-04-15T15:18:44.861740+0000","last_unstale":"2024-04-15T15:18:46.874502+0000","last_undegraded":"2024-04-15T15:18:46.874502+0000","last_fullsized":"2024-04-15T15:18:46.874502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.848791+0000","last_change":"2024-04-15T15:18:44.867636+0000","last_active":"2024-04-15T15:18:45.848791+0000","last_peered":"2024-04-15T15:18:45.848791+0000","last_clean":"2024-04-15T15:18:45.848791+0000","last_became_active":"2024-04-15T15:18:44.867307+0000","last_became_peered":"2024-04-15T15:18:44.867307+0000","last_unstale":"2024-04-15T15:18:45.848791+0000","last_undegraded":"2024-04-15T15:18:45.848791+0000","last_fullsized":"2024-04-15T15:18:45.848791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858064+0000","last_change":"2024-04-15T15:18:45.850873+0000","last_active":"2024-04-15T15:18:45.858064+0000","last_peered":"2024-04-15T15:18:45.858064+0000","last_clean":"2024-04-15T15:18:45.858064+0000","last_became_active":"2024-04-15T15:18:45.850532+0000","last_became_peered":"2024-04-15T15:18:45.850532+0000","last_unstale":"2024-04-15T15:18:45.858064+0000","last_undegraded":"2024-04-15T15:18:45.858064+0000","last_fullsized":"2024-04-15T15:18:45.858064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:22:54.875123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821134+0000","last_change":"2024-04-15T15:18:44.866286+0000","last_active":"2024-04-15T15:18:46.821134+0000","last_peered":"2024-04-15T15:18:46.821134+0000","last_clean":"2024-04-15T15:18:46.821134+0000","last_became_active":"2024-04-15T15:18:44.863387+0000","last_became_peered":"2024-04-15T15:18:44.863387+0000","last_unstale":"2024-04-15T15:18:46.821134+0000","last_undegraded":"2024-04-15T15:18:46.821134+0000","last_fullsized":"2024-04-15T15:18:46.821134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869851+0000","last_change":"2024-04-15T15:18:45.839256+0000","last_active":"2024-04-15T15:18:46.869851+0000","last_peered":"2024-04-15T15:18:46.869851+0000","last_clean":"2024-04-15T15:18:46.869851+0000","last_became_active":"2024-04-15T15:18:45.838960+0000","last_became_peered":"2024-04-15T15:18:45.838960+0000","last_unstale":"2024-04-15T15:18:46.869851+0000","last_undegraded":"2024-04-15T15:18:46.869851+0000","last_fullsized":"2024-04-15T15:18:46.869851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875122+0000","last_change":"2024-04-15T15:18:44.870228+0000","last_active":"2024-04-15T15:18:46.875122+0000","last_peered":"2024-04-15T15:18:46.875122+0000","last_clean":"2024-04-15T15:18:46.875122+0000","last_became_active":"2024-04-15T15:18:44.869155+0000","last_became_peered":"2024-04-15T15:18:44.869155+0000","last_unstale":"2024-04-15T15:18:46.875122+0000","last_undegraded":"2024-04-15T15:18:46.875122+0000","last_fullsized":"2024-04-15T15:18:46.875122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824756+0000","last_change":"2024-04-15T15:18:45.845321+0000","last_active":"2024-04-15T15:18:46.824756+0000","last_peered":"2024-04-15T15:18:46.824756+0000","last_clean":"2024-04-15T15:18:46.824756+0000","last_became_active":"2024-04-15T15:18:45.840422+0000","last_became_peered":"2024-04-15T15:18:45.840422+0000","last_unstale":"2024-04-15T15:18:46.824756+0000","last_undegraded":"2024-04-15T15:18:46.824756+0000","last_fullsized":"2024-04-15T15:18:46.824756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:32:58.852361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824852+0000","last_change":"2024-04-15T15:18:44.865670+0000","last_active":"2024-04-15T15:18:46.824852+0000","last_peered":"2024-04-15T15:18:46.824852+0000","last_clean":"2024-04-15T15:18:46.824852+0000","last_became_active":"2024-04-15T15:18:44.864795+0000","last_became_peered":"2024-04-15T15:18:44.864795+0000","last_unstale":"2024-04-15T15:18:46.824852+0000","last_undegraded":"2024-04-15T15:18:46.824852+0000","last_fullsized":"2024-04-15T15:18:46.824852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.855281+0000","last_change":"2024-04-15T15:18:45.852169+0000","last_active":"2024-04-15T15:18:45.855281+0000","last_peered":"2024-04-15T15:18:45.855281+0000","last_clean":"2024-04-15T15:18:45.855281+0000","last_became_active":"2024-04-15T15:18:45.845297+0000","last_became_peered":"2024-04-15T15:18:45.845297+0000","last_unstale":"2024-04-15T15:18:45.855281+0000","last_undegraded":"2024-04-15T15:18:45.855281+0000","last_fullsized":"2024-04-15T15:18:45.855281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:55.246279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870182+0000","last_change":"2024-04-15T15:18:44.866158+0000","last_active":"2024-04-15T15:18:46.870182+0000","last_peered":"2024-04-15T15:18:46.870182+0000","last_clean":"2024-04-15T15:18:46.870182+0000","last_became_active":"2024-04-15T15:18:44.865766+0000","last_became_peered":"2024-04-15T15:18:44.865766+0000","last_unstale":"2024-04-15T15:18:46.870182+0000","last_undegraded":"2024-04-15T15:18:46.870182+0000","last_fullsized":"2024-04-15T15:18:46.870182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822687+0000","last_change":"2024-04-15T15:18:45.843966+0000","last_active":"2024-04-15T15:18:46.822687+0000","last_peered":"2024-04-15T15:18:46.822687+0000","last_clean":"2024-04-15T15:18:46.822687+0000","last_became_active":"2024-04-15T15:18:45.842648+0000","last_became_peered":"2024-04-15T15:18:45.842648+0000","last_unstale":"2024-04-15T15:18:46.822687+0000","last_undegraded":"2024-04-15T15:18:46.822687+0000","last_fullsized":"2024-04-15T15:18:46.822687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:39:50.034981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874917+0000","last_change":"2024-04-15T15:18:44.867158+0000","last_active":"2024-04-15T15:18:46.874917+0000","last_peered":"2024-04-15T15:18:46.874917+0000","last_clean":"2024-04-15T15:18:46.874917+0000","last_became_active":"2024-04-15T15:18:44.866840+0000","last_became_peered":"2024-04-15T15:18:44.866840+0000","last_unstale":"2024-04-15T15:18:46.874917+0000","last_undegraded":"2024-04-15T15:18:46.874917+0000","last_fullsized":"2024-04-15T15:18:46.874917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818675+0000","last_change":"2024-04-15T15:18:45.860752+0000","last_active":"2024-04-15T15:18:46.818675+0000","last_peered":"2024-04-15T15:18:46.818675+0000","last_clean":"2024-04-15T15:18:46.818675+0000","last_became_active":"2024-04-15T15:18:45.860381+0000","last_became_peered":"2024-04-15T15:18:45.860381+0000","last_unstale":"2024-04-15T15:18:46.818675+0000","last_undegraded":"2024-04-15T15:18:46.818675+0000","last_fullsized":"2024-04-15T15:18:46.818675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:53.984837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.875312+0000","last_change":"2024-04-15T15:18:44.865018+0000","last_active":"2024-04-15T15:18:45.875312+0000","last_peered":"2024-04-15T15:18:45.875312+0000","last_clean":"2024-04-15T15:18:45.875312+0000","last_became_active":"2024-04-15T15:18:44.864073+0000","last_became_peered":"2024-04-15T15:18:44.864073+0000","last_unstale":"2024-04-15T15:18:45.875312+0000","last_undegraded":"2024-04-15T15:18:45.875312+0000","last_fullsized":"2024-04-15T15:18:45.875312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:34.796362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.861410+0000","last_change":"2024-04-15T15:18:45.860189+0000","last_active":"2024-04-15T15:18:45.861410+0000","last_peered":"2024-04-15T15:18:45.861410+0000","last_clean":"2024-04-15T15:18:45.861410+0000","last_became_active":"2024-04-15T15:18:45.859786+0000","last_became_peered":"2024-04-15T15:18:45.859786+0000","last_unstale":"2024-04-15T15:18:45.861410+0000","last_undegraded":"2024-04-15T15:18:45.861410+0000","last_fullsized":"2024-04-15T15:18:45.861410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:18.727426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822942+0000","last_change":"2024-04-15T15:18:44.866125+0000","last_active":"2024-04-15T15:18:46.822942+0000","last_peered":"2024-04-15T15:18:46.822942+0000","last_clean":"2024-04-15T15:18:46.822942+0000","last_became_active":"2024-04-15T15:18:44.864849+0000","last_became_peered":"2024-04-15T15:18:44.864849+0000","last_unstale":"2024-04-15T15:18:46.822942+0000","last_undegraded":"2024-04-15T15:18:46.822942+0000","last_fullsized":"2024-04-15T15:18:46.822942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819575+0000","last_change":"2024-04-15T15:18:45.852862+0000","last_active":"2024-04-15T15:18:46.819575+0000","last_peered":"2024-04-15T15:18:46.819575+0000","last_clean":"2024-04-15T15:18:46.819575+0000","last_became_active":"2024-04-15T15:18:45.851610+0000","last_became_peered":"2024-04-15T15:18:45.851610+0000","last_unstale":"2024-04-15T15:18:46.819575+0000","last_undegraded":"2024-04-15T15:18:46.819575+0000","last_fullsized":"2024-04-15T15:18:46.819575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:36.969295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874519+0000","last_change":"2024-04-15T15:18:44.871366+0000","last_active":"2024-04-15T15:18:46.874519+0000","last_peered":"2024-04-15T15:18:46.874519+0000","last_clean":"2024-04-15T15:18:46.874519+0000","last_became_active":"2024-04-15T15:18:44.870865+0000","last_became_peered":"2024-04-15T15:18:44.870865+0000","last_unstale":"2024-04-15T15:18:46.874519+0000","last_undegraded":"2024-04-15T15:18:46.874519+0000","last_fullsized":"2024-04-15T15:18:46.874519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874458+0000","last_change":"2024-04-15T15:18:45.847642+0000","last_active":"2024-04-15T15:18:46.874458+0000","last_peered":"2024-04-15T15:18:46.874458+0000","last_clean":"2024-04-15T15:18:46.874458+0000","last_became_active":"2024-04-15T15:18:45.845790+0000","last_became_peered":"2024-04-15T15:18:45.845790+0000","last_unstale":"2024-04-15T15:18:46.874458+0000","last_undegraded":"2024-04-15T15:18:46.874458+0000","last_fullsized":"2024-04-15T15:18:46.874458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.878253+0000","last_change":"2024-04-15T15:18:44.857029+0000","last_active":"2024-04-15T15:18:45.878253+0000","last_peered":"2024-04-15T15:18:45.878253+0000","last_clean":"2024-04-15T15:18:45.878253+0000","last_became_active":"2024-04-15T15:18:44.855203+0000","last_became_peered":"2024-04-15T15:18:44.855203+0000","last_unstale":"2024-04-15T15:18:45.878253+0000","last_undegraded":"2024-04-15T15:18:45.878253+0000","last_fullsized":"2024-04-15T15:18:45.878253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:34.796362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818209+0000","last_change":"2024-04-15T15:18:45.856661+0000","last_active":"2024-04-15T15:18:46.818209+0000","last_peered":"2024-04-15T15:18:46.818209+0000","last_clean":"2024-04-15T15:18:46.818209+0000","last_became_active":"2024-04-15T15:18:45.856443+0000","last_became_peered":"2024-04-15T15:18:45.856443+0000","last_unstale":"2024-04-15T15:18:46.818209+0000","last_undegraded":"2024-04-15T15:18:46.818209+0000","last_fullsized":"2024-04-15T15:18:46.818209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.566254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822757+0000","last_change":"2024-04-15T15:18:44.866401+0000","last_active":"2024-04-15T15:18:46.822757+0000","last_peered":"2024-04-15T15:18:46.822757+0000","last_clean":"2024-04-15T15:18:46.822757+0000","last_became_active":"2024-04-15T15:18:44.866099+0000","last_became_peered":"2024-04-15T15:18:44.866099+0000","last_unstale":"2024-04-15T15:18:46.822757+0000","last_undegraded":"2024-04-15T15:18:46.822757+0000","last_fullsized":"2024-04-15T15:18:46.822757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:15:03.621969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820130+0000","last_change":"2024-04-15T15:18:45.854183+0000","last_active":"2024-04-15T15:18:46.820130+0000","last_peered":"2024-04-15T15:18:46.820130+0000","last_clean":"2024-04-15T15:18:46.820130+0000","last_became_active":"2024-04-15T15:18:45.853850+0000","last_became_peered":"2024-04-15T15:18:45.853850+0000","last_unstale":"2024-04-15T15:18:46.820130+0000","last_undegraded":"2024-04-15T15:18:46.820130+0000","last_fullsized":"2024-04-15T15:18:46.820130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:32:58.852361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.876882+0000","last_change":"2024-04-15T15:18:44.857499+0000","last_active":"2024-04-15T15:18:45.876882+0000","last_peered":"2024-04-15T15:18:45.876882+0000","last_clean":"2024-04-15T15:18:45.876882+0000","last_became_active":"2024-04-15T15:18:44.857140+0000","last_became_peered":"2024-04-15T15:18:44.857140+0000","last_unstale":"2024-04-15T15:18:45.876882+0000","last_undegraded":"2024-04-15T15:18:45.876882+0000","last_fullsized":"2024-04-15T15:18:45.876882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.826627+0000","last_change":"2024-04-15T15:18:45.860607+0000","last_active":"2024-04-15T15:18:46.826627+0000","last_peered":"2024-04-15T15:18:46.826627+0000","last_clean":"2024-04-15T15:18:46.826627+0000","last_became_active":"2024-04-15T15:18:45.858730+0000","last_became_peered":"2024-04-15T15:18:45.858730+0000","last_unstale":"2024-04-15T15:18:46.826627+0000","last_undegraded":"2024-04-15T15:18:46.826627+0000","last_fullsized":"2024-04-15T15:18:46.826627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:10:16.717594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870214+0000","last_change":"2024-04-15T15:18:44.858372+0000","last_active":"2024-04-15T15:18:46.870214+0000","last_peered":"2024-04-15T15:18:46.870214+0000","last_clean":"2024-04-15T15:18:46.870214+0000","last_became_active":"2024-04-15T15:18:44.858083+0000","last_became_peered":"2024-04-15T15:18:44.858083+0000","last_unstale":"2024-04-15T15:18:46.870214+0000","last_undegraded":"2024-04-15T15:18:46.870214+0000","last_fullsized":"2024-04-15T15:18:46.870214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824118+0000","last_change":"2024-04-15T15:18:45.864746+0000","last_active":"2024-04-15T15:18:46.824118+0000","last_peered":"2024-04-15T15:18:46.824118+0000","last_clean":"2024-04-15T15:18:46.824118+0000","last_became_active":"2024-04-15T15:18:45.858273+0000","last_became_peered":"2024-04-15T15:18:45.858273+0000","last_unstale":"2024-04-15T15:18:46.824118+0000","last_undegraded":"2024-04-15T15:18:46.824118+0000","last_fullsized":"2024-04-15T15:18:46.824118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:36.969295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.835083+0000","last_change":"2024-04-15T15:18:44.864232+0000","last_active":"2024-04-15T15:18:45.835083+0000","last_peered":"2024-04-15T15:18:45.835083+0000","last_clean":"2024-04-15T15:18:45.835083+0000","last_became_active":"2024-04-15T15:18:44.863837+0000","last_became_peered":"2024-04-15T15:18:44.863837+0000","last_unstale":"2024-04-15T15:18:45.835083+0000","last_undegraded":"2024-04-15T15:18:45.835083+0000","last_fullsized":"2024-04-15T15:18:45.835083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818747+0000","last_change":"2024-04-15T15:18:45.860687+0000","last_active":"2024-04-15T15:18:46.818747+0000","last_peered":"2024-04-15T15:18:46.818747+0000","last_clean":"2024-04-15T15:18:46.818747+0000","last_became_active":"2024-04-15T15:18:45.860227+0000","last_became_peered":"2024-04-15T15:18:45.860227+0000","last_unstale":"2024-04-15T15:18:46.818747+0000","last_undegraded":"2024-04-15T15:18:46.818747+0000","last_fullsized":"2024-04-15T15:18:46.818747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:29:14.132066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.817987+0000","last_change":"2024-04-15T15:18:41.162887+0000","last_active":"2024-04-15T15:18:46.817987+0000","last_peered":"2024-04-15T15:18:46.817987+0000","last_clean":"2024-04-15T15:18:46.817987+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:18:46.817987+0000","last_undegraded":"2024-04-15T15:18:46.817987+0000","last_fullsized":"2024-04-15T15:18:46.817987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_clean_scrub_stamp":"2024-04-15T15:18:39.644354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:32:00.047414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821321+0000","last_change":"2024-04-15T15:18:44.858206+0000","last_active":"2024-04-15T15:18:46.821321+0000","last_peered":"2024-04-15T15:18:46.821321+0000","last_clean":"2024-04-15T15:18:46.821321+0000","last_became_active":"2024-04-15T15:18:44.857694+0000","last_became_peered":"2024-04-15T15:18:44.857694+0000","last_unstale":"2024-04-15T15:18:46.821321+0000","last_undegraded":"2024-04-15T15:18:46.821321+0000","last_fullsized":"2024-04-15T15:18:46.821321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.857312+0000","last_change":"2024-04-15T15:18:45.849224+0000","last_active":"2024-04-15T15:18:45.857312+0000","last_peered":"2024-04-15T15:18:45.857312+0000","last_clean":"2024-04-15T15:18:45.857312+0000","last_became_active":"2024-04-15T15:18:45.848961+0000","last_became_peered":"2024-04-15T15:18:45.848961+0000","last_unstale":"2024-04-15T15:18:45.857312+0000","last_undegraded":"2024-04-15T15:18:45.857312+0000","last_fullsized":"2024-04-15T15:18:45.857312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:18.082210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.817850+0000","last_change":"2024-04-15T15:18:44.859202+0000","last_active":"2024-04-15T15:18:46.817850+0000","last_peered":"2024-04-15T15:18:46.817850+0000","last_clean":"2024-04-15T15:18:46.817850+0000","last_became_active":"2024-04-15T15:18:44.858791+0000","last_became_peered":"2024-04-15T15:18:44.858791+0000","last_unstale":"2024-04-15T15:18:46.817850+0000","last_undegraded":"2024-04-15T15:18:46.817850+0000","last_fullsized":"2024-04-15T15:18:46.817850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.854716+0000","last_change":"2024-04-15T15:18:45.851981+0000","last_active":"2024-04-15T15:18:45.854716+0000","last_peered":"2024-04-15T15:18:45.854716+0000","last_clean":"2024-04-15T15:18:45.854716+0000","last_became_active":"2024-04-15T15:18:45.851653+0000","last_became_peered":"2024-04-15T15:18:45.851653+0000","last_unstale":"2024-04-15T15:18:45.854716+0000","last_undegraded":"2024-04-15T15:18:45.854716+0000","last_fullsized":"2024-04-15T15:18:45.854716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:51.536346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.833897+0000","last_change":"2024-04-15T15:18:44.849893+0000","last_active":"2024-04-15T15:18:45.833897+0000","last_peered":"2024-04-15T15:18:45.833897+0000","last_clean":"2024-04-15T15:18:45.833897+0000","last_became_active":"2024-04-15T15:18:44.848047+0000","last_became_peered":"2024-04-15T15:18:44.848047+0000","last_unstale":"2024-04-15T15:18:45.833897+0000","last_undegraded":"2024-04-15T15:18:45.833897+0000","last_fullsized":"2024-04-15T15:18:45.833897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869862+0000","last_change":"2024-04-15T15:18:45.844515+0000","last_active":"2024-04-15T15:18:46.869862+0000","last_peered":"2024-04-15T15:18:46.869862+0000","last_clean":"2024-04-15T15:18:46.869862+0000","last_became_active":"2024-04-15T15:18:45.844154+0000","last_became_peered":"2024-04-15T15:18:45.844154+0000","last_unstale":"2024-04-15T15:18:46.869862+0000","last_undegraded":"2024-04-15T15:18:46.869862+0000","last_fullsized":"2024-04-15T15:18:46.869862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823178+0000","last_change":"2024-04-15T15:18:44.862254+0000","last_active":"2024-04-15T15:18:46.823178+0000","last_peered":"2024-04-15T15:18:46.823178+0000","last_clean":"2024-04-15T15:18:46.823178+0000","last_became_active":"2024-04-15T15:18:44.861231+0000","last_became_peered":"2024-04-15T15:18:44.861231+0000","last_unstale":"2024-04-15T15:18:46.823178+0000","last_undegraded":"2024-04-15T15:18:46.823178+0000","last_fullsized":"2024-04-15T15:18:46.823178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821451+0000","last_change":"2024-04-15T15:18:45.847966+0000","last_active":"2024-04-15T15:18:46.821451+0000","last_peered":"2024-04-15T15:18:46.821451+0000","last_clean":"2024-04-15T15:18:46.821451+0000","last_became_active":"2024-04-15T15:18:45.847628+0000","last_became_peered":"2024-04-15T15:18:45.847628+0000","last_unstale":"2024-04-15T15:18:46.821451+0000","last_undegraded":"2024-04-15T15:18:46.821451+0000","last_fullsized":"2024-04-15T15:18:46.821451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:35:55.040215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870108+0000","last_change":"2024-04-15T15:18:44.866681+0000","last_active":"2024-04-15T15:18:46.870108+0000","last_peered":"2024-04-15T15:18:46.870108+0000","last_clean":"2024-04-15T15:18:46.870108+0000","last_became_active":"2024-04-15T15:18:44.866366+0000","last_became_peered":"2024-04-15T15:18:44.866366+0000","last_unstale":"2024-04-15T15:18:46.870108+0000","last_undegraded":"2024-04-15T15:18:46.870108+0000","last_fullsized":"2024-04-15T15:18:46.870108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824652+0000","last_change":"2024-04-15T15:18:45.858129+0000","last_active":"2024-04-15T15:18:46.824652+0000","last_peered":"2024-04-15T15:18:46.824652+0000","last_clean":"2024-04-15T15:18:46.824652+0000","last_became_active":"2024-04-15T15:18:45.857824+0000","last_became_peered":"2024-04-15T15:18:45.857824+0000","last_unstale":"2024-04-15T15:18:46.824652+0000","last_undegraded":"2024-04-15T15:18:46.824652+0000","last_fullsized":"2024-04-15T15:18:46.824652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:36:03.614062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820284+0000","last_change":"2024-04-15T15:18:44.866754+0000","last_active":"2024-04-15T15:18:46.820284+0000","last_peered":"2024-04-15T15:18:46.820284+0000","last_clean":"2024-04-15T15:18:46.820284+0000","last_became_active":"2024-04-15T15:18:44.866479+0000","last_became_peered":"2024-04-15T15:18:44.866479+0000","last_unstale":"2024-04-15T15:18:46.820284+0000","last_undegraded":"2024-04-15T15:18:46.820284+0000","last_fullsized":"2024-04-15T15:18:46.820284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:17:09.815187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870141+0000","last_change":"2024-04-15T15:18:45.844997+0000","last_active":"2024-04-15T15:18:46.870141+0000","last_peered":"2024-04-15T15:18:46.870141+0000","last_clean":"2024-04-15T15:18:46.870141+0000","last_became_active":"2024-04-15T15:18:45.844589+0000","last_became_peered":"2024-04-15T15:18:45.844589+0000","last_unstale":"2024-04-15T15:18:46.870141+0000","last_undegraded":"2024-04-15T15:18:46.870141+0000","last_fullsized":"2024-04-15T15:18:46.870141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.873141+0000","last_change":"2024-04-15T15:18:44.861663+0000","last_active":"2024-04-15T15:18:46.873141+0000","last_peered":"2024-04-15T15:18:46.873141+0000","last_clean":"2024-04-15T15:18:46.873141+0000","last_became_active":"2024-04-15T15:18:44.861347+0000","last_became_peered":"2024-04-15T15:18:44.861347+0000","last_unstale":"2024-04-15T15:18:46.873141+0000","last_undegraded":"2024-04-15T15:18:46.873141+0000","last_fullsized":"2024-04-15T15:18:46.873141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:54.111084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858205+0000","last_change":"2024-04-15T15:18:45.854665+0000","last_active":"2024-04-15T15:18:45.858205+0000","last_peered":"2024-04-15T15:18:45.858205+0000","last_clean":"2024-04-15T15:18:45.858205+0000","last_became_active":"2024-04-15T15:18:45.853958+0000","last_became_peered":"2024-04-15T15:18:45.853958+0000","last_unstale":"2024-04-15T15:18:45.858205+0000","last_undegraded":"2024-04-15T15:18:45.858205+0000","last_fullsized":"2024-04-15T15:18:45.858205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:57:54.115969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875345+0000","last_change":"2024-04-15T15:18:44.866078+0000","last_active":"2024-04-15T15:18:46.875345+0000","last_peered":"2024-04-15T15:18:46.875345+0000","last_clean":"2024-04-15T15:18:46.875345+0000","last_became_active":"2024-04-15T15:18:44.865730+0000","last_became_peered":"2024-04-15T15:18:44.865730+0000","last_unstale":"2024-04-15T15:18:46.875345+0000","last_undegraded":"2024-04-15T15:18:46.875345+0000","last_fullsized":"2024-04-15T15:18:46.875345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.844098+0000","last_change":"2024-04-15T15:18:45.841162+0000","last_active":"2024-04-15T15:18:45.844098+0000","last_peered":"2024-04-15T15:18:45.844098+0000","last_clean":"2024-04-15T15:18:45.844098+0000","last_became_active":"2024-04-15T15:18:45.840894+0000","last_became_peered":"2024-04-15T15:18:45.840894+0000","last_unstale":"2024-04-15T15:18:45.844098+0000","last_undegraded":"2024-04-15T15:18:45.844098+0000","last_fullsized":"2024-04-15T15:18:45.844098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:18.082210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819201+0000","last_change":"2024-04-15T15:18:44.862312+0000","last_active":"2024-04-15T15:18:46.819201+0000","last_peered":"2024-04-15T15:18:46.819201+0000","last_clean":"2024-04-15T15:18:46.819201+0000","last_became_active":"2024-04-15T15:18:44.862026+0000","last_became_peered":"2024-04-15T15:18:44.862026+0000","last_unstale":"2024-04-15T15:18:46.819201+0000","last_undegraded":"2024-04-15T15:18:46.819201+0000","last_fullsized":"2024-04-15T15:18:46.819201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822643+0000","last_change":"2024-04-15T15:18:45.847316+0000","last_active":"2024-04-15T15:18:46.822643+0000","last_peered":"2024-04-15T15:18:46.822643+0000","last_clean":"2024-04-15T15:18:46.822643+0000","last_became_active":"2024-04-15T15:18:45.846987+0000","last_became_peered":"2024-04-15T15:18:45.846987+0000","last_unstale":"2024-04-15T15:18:46.822643+0000","last_undegraded":"2024-04-15T15:18:46.822643+0000","last_fullsized":"2024-04-15T15:18:46.822643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:58:24.586889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869976+0000","last_change":"2024-04-15T15:18:44.859986+0000","last_active":"2024-04-15T15:18:46.869976+0000","last_peered":"2024-04-15T15:18:46.869976+0000","last_clean":"2024-04-15T15:18:46.869976+0000","last_became_active":"2024-04-15T15:18:44.859733+0000","last_became_peered":"2024-04-15T15:18:44.859733+0000","last_unstale":"2024-04-15T15:18:46.869976+0000","last_undegraded":"2024-04-15T15:18:46.869976+0000","last_fullsized":"2024-04-15T15:18:46.869976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:26.555787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.855272+0000","last_change":"2024-04-15T15:18:45.846102+0000","last_active":"2024-04-15T15:18:45.855272+0000","last_peered":"2024-04-15T15:18:45.855272+0000","last_clean":"2024-04-15T15:18:45.855272+0000","last_became_active":"2024-04-15T15:18:45.845715+0000","last_became_peered":"2024-04-15T15:18:45.845715+0000","last_unstale":"2024-04-15T15:18:45.855272+0000","last_undegraded":"2024-04-15T15:18:45.855272+0000","last_fullsized":"2024-04-15T15:18:45.855272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:50:45.496232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869992+0000","last_change":"2024-04-15T15:18:44.862018+0000","last_active":"2024-04-15T15:18:46.869992+0000","last_peered":"2024-04-15T15:18:46.869992+0000","last_clean":"2024-04-15T15:18:46.869992+0000","last_became_active":"2024-04-15T15:18:44.861663+0000","last_became_peered":"2024-04-15T15:18:44.861663+0000","last_unstale":"2024-04-15T15:18:46.869992+0000","last_undegraded":"2024-04-15T15:18:46.869992+0000","last_fullsized":"2024-04-15T15:18:46.869992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.858232+0000","last_change":"2024-04-15T15:18:45.852079+0000","last_active":"2024-04-15T15:18:45.858232+0000","last_peered":"2024-04-15T15:18:45.858232+0000","last_clean":"2024-04-15T15:18:45.858232+0000","last_became_active":"2024-04-15T15:18:45.851281+0000","last_became_peered":"2024-04-15T15:18:45.851281+0000","last_unstale":"2024-04-15T15:18:45.858232+0000","last_undegraded":"2024-04-15T15:18:45.858232+0000","last_fullsized":"2024-04-15T15:18:45.858232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:55.246279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819899+0000","last_change":"2024-04-15T15:18:44.867447+0000","last_active":"2024-04-15T15:18:46.819899+0000","last_peered":"2024-04-15T15:18:46.819899+0000","last_clean":"2024-04-15T15:18:46.819899+0000","last_became_active":"2024-04-15T15:18:44.867171+0000","last_became_peered":"2024-04-15T15:18:44.867171+0000","last_unstale":"2024-04-15T15:18:46.819899+0000","last_undegraded":"2024-04-15T15:18:46.819899+0000","last_fullsized":"2024-04-15T15:18:46.819899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:26.555787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874593+0000","last_change":"2024-04-15T15:18:45.861329+0000","last_active":"2024-04-15T15:18:46.874593+0000","last_peered":"2024-04-15T15:18:46.874593+0000","last_clean":"2024-04-15T15:18:46.874593+0000","last_became_active":"2024-04-15T15:18:45.859564+0000","last_became_peered":"2024-04-15T15:18:45.859564+0000","last_unstale":"2024-04-15T15:18:46.874593+0000","last_undegraded":"2024-04-15T15:18:46.874593+0000","last_fullsized":"2024-04-15T15:18:46.874593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824729+0000","last_change":"2024-04-15T15:18:45.831940+0000","last_active":"2024-04-15T15:18:46.824729+0000","last_peered":"2024-04-15T15:18:46.824729+0000","last_clean":"2024-04-15T15:18:46.824729+0000","last_became_active":"2024-04-15T15:18:45.829915+0000","last_became_peered":"2024-04-15T15:18:45.829915+0000","last_unstale":"2024-04-15T15:18:46.824729+0000","last_undegraded":"2024-04-15T15:18:46.824729+0000","last_fullsized":"2024-04-15T15:18:46.824729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:06:25.954239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818726+0000","last_change":"2024-04-15T15:18:44.869358+0000","last_active":"2024-04-15T15:18:46.818726+0000","last_peered":"2024-04-15T15:18:46.818726+0000","last_clean":"2024-04-15T15:18:46.818726+0000","last_became_active":"2024-04-15T15:18:44.869040+0000","last_became_peered":"2024-04-15T15:18:44.869040+0000","last_unstale":"2024-04-15T15:18:46.818726+0000","last_undegraded":"2024-04-15T15:18:46.818726+0000","last_fullsized":"2024-04-15T15:18:46.818726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:54.111084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823654+0000","last_change":"2024-04-15T15:18:45.851111+0000","last_active":"2024-04-15T15:18:46.823654+0000","last_peered":"2024-04-15T15:18:46.823654+0000","last_clean":"2024-04-15T15:18:46.823654+0000","last_became_active":"2024-04-15T15:18:45.850824+0000","last_became_peered":"2024-04-15T15:18:45.850824+0000","last_unstale":"2024-04-15T15:18:46.823654+0000","last_undegraded":"2024-04-15T15:18:46.823654+0000","last_fullsized":"2024-04-15T15:18:46.823654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:53.984837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818964+0000","last_change":"2024-04-15T15:18:44.864267+0000","last_active":"2024-04-15T15:18:46.818964+0000","last_peered":"2024-04-15T15:18:46.818964+0000","last_clean":"2024-04-15T15:18:46.818964+0000","last_became_active":"2024-04-15T15:18:44.863962+0000","last_became_peered":"2024-04-15T15:18:44.863962+0000","last_unstale":"2024-04-15T15:18:46.818964+0000","last_undegraded":"2024-04-15T15:18:46.818964+0000","last_fullsized":"2024-04-15T15:18:46.818964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:22:53.739928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.852693+0000","last_change":"2024-04-15T15:18:45.845359+0000","last_active":"2024-04-15T15:18:45.852693+0000","last_peered":"2024-04-15T15:18:45.852693+0000","last_clean":"2024-04-15T15:18:45.852693+0000","last_became_active":"2024-04-15T15:18:45.845087+0000","last_became_peered":"2024-04-15T15:18:45.845087+0000","last_unstale":"2024-04-15T15:18:45.852693+0000","last_undegraded":"2024-04-15T15:18:45.852693+0000","last_fullsized":"2024-04-15T15:18:45.852693+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:20.202506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.875540+0000","last_change":"2024-04-15T15:18:44.864497+0000","last_active":"2024-04-15T15:18:45.875540+0000","last_peered":"2024-04-15T15:18:45.875540+0000","last_clean":"2024-04-15T15:18:45.875540+0000","last_became_active":"2024-04-15T15:18:44.864122+0000","last_became_peered":"2024-04-15T15:18:44.864122+0000","last_unstale":"2024-04-15T15:18:45.875540+0000","last_undegraded":"2024-04-15T15:18:45.875540+0000","last_fullsized":"2024-04-15T15:18:45.875540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.863115+0000","last_change":"2024-04-15T15:18:45.861978+0000","last_active":"2024-04-15T15:18:45.863115+0000","last_peered":"2024-04-15T15:18:45.863115+0000","last_clean":"2024-04-15T15:18:45.863115+0000","last_became_active":"2024-04-15T15:18:45.861597+0000","last_became_peered":"2024-04-15T15:18:45.861597+0000","last_unstale":"2024-04-15T15:18:45.863115+0000","last_undegraded":"2024-04-15T15:18:45.863115+0000","last_fullsized":"2024-04-15T15:18:45.863115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:16:52.507783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-15T15:18:45.879016+0000","last_change":"2024-04-15T15:18:44.864325+0000","last_active":"2024-04-15T15:18:45.879016+0000","last_peered":"2024-04-15T15:18:45.879016+0000","last_clean":"2024-04-15T15:18:45.879016+0000","last_became_active":"2024-04-15T15:18:44.863985+0000","last_became_peered":"2024-04-15T15:18:44.863985+0000","last_unstale":"2024-04-15T15:18:45.879016+0000","last_undegraded":"2024-04-15T15:18:45.879016+0000","last_fullsized":"2024-04-15T15:18:45.879016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":811008,"data_stored":663558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":72734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":74220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":72608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":79068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":819200,"data_stored":663614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:18:50.899 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-15T15:18:50.899 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-15T15:18:50.899 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-15T15:18:50.900 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-15T15:18:50.900 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-15T15:18:51.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.168+0000 7f996fe00640 1 Processor -- start 2024-04-15T15:18:51.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.169+0000 7f996fe00640 1 -- start start 2024-04-15T15:18:51.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.170+0000 7f996fe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99700d1c90 0x7f9970162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.170+0000 7f996fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99700cf110 0x7f99700d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.170+0000 7f996fe00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99700d0090 con 0x7f9970162020 2024-04-15T15:18:51.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.170+0000 7f996fe00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f9970163440 con 0x7f99700d1c90 2024-04-15T15:18:51.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.170+0000 7f996fe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f99700b7b30 con 0x7f99700cf110 2024-04-15T15:18:51.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.170+0000 7f996e000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99700d1c90 0x7f9970162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.171+0000 7f996e000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99700d1c90 0x7f9970162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58332/0 (socket says 172.21.15.49:58332) 2024-04-15T15:18:51.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.171+0000 7f996e000640 1 -- 172.21.15.49:0/2101244732 learned_addr learned my addr 172.21.15.49:0/2101244732 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:51.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.171+0000 7f996f400640 1 --2- 172.21.15.49:0/2101244732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99700cf110 0x7f99700d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.171+0000 7f996e000640 1 -- 172.21.15.49:0/2101244732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99700cf110 msgr2=0x7f99700d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:51.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.171+0000 7f996e000640 1 --2- 172.21.15.49:0/2101244732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99700cf110 0x7f99700d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.171+0000 7f996e000640 1 -- 172.21.15.49:0/2101244732 >> v1:172.21.15.107:6789/0 conn(0x7f9970162020 legacy=0x7f99700d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:51.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.171+0000 7f996e000640 1 -- 172.21.15.49:0/2101244732 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99700cfad0 con 0x7f99700d1c90 2024-04-15T15:18:51.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.172+0000 7f996e000640 1 --2- 172.21.15.49:0/2101244732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99700d1c90 0x7f9970162f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9958009870 tx=0x7f995802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783b956ebeb2c2a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:51.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.173+0000 7f996d600640 1 -- 172.21.15.49:0/2101244732 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958035020 con 0x7f99700d1c90 2024-04-15T15:18:51.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.173+0000 7f996d600640 1 -- 172.21.15.49:0/2101244732 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9958031640 con 0x7f99700d1c90 2024-04-15T15:18:51.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.173+0000 7f996d600640 1 -- 172.21.15.49:0/2101244732 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995802d040 con 0x7f99700d1c90 2024-04-15T15:18:51.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.174+0000 7f996fe00640 1 -- 172.21.15.49:0/2101244732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99700d1c90 msgr2=0x7f9970162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.174+0000 7f996fe00640 1 --2- 172.21.15.49:0/2101244732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99700d1c90 0x7f9970162f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9958009870 tx=0x7f995802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.174+0000 7f996fe00640 1 -- 172.21.15.49:0/2101244732 shutdown_connections 2024-04-15T15:18:51.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.174+0000 7f996fe00640 1 --2- 172.21.15.49:0/2101244732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99700cf110 0x7f99700d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.174+0000 7f996fe00640 1 --2- 172.21.15.49:0/2101244732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99700d1c90 0x7f9970162f00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.174+0000 7f996fe00640 1 -- 172.21.15.49:0/2101244732 >> 172.21.15.49:0/2101244732 conn(0x7f997006cf00 msgr2=0x7f997006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:51.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.174+0000 7f996fe00640 1 -- 172.21.15.49:0/2101244732 shutdown_connections 2024-04-15T15:18:51.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.174+0000 7f996fe00640 1 -- 172.21.15.49:0/2101244732 wait complete. 2024-04-15T15:18:51.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.176+0000 7f996fe00640 1 Processor -- start 2024-04-15T15:18:51.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.176+0000 7f996fe00640 1 -- start start 2024-04-15T15:18:51.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.177+0000 7f996fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99700cf110 0x7f99701e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.177+0000 7f996fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99700d1c90 0x7f99700ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.177+0000 7f996fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9970162020 0x7f99700cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.177+0000 7f996fe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f99700d2b90 con 0x7f9970162020 2024-04-15T15:18:51.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.177+0000 7f996fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f99700cf880 con 0x7f99700cf110 2024-04-15T15:18:51.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.177+0000 7f996fe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f99700d2e70 con 0x7f99700d1c90 2024-04-15T15:18:51.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9970162020 0x7f99700cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9970162020 0x7f99700cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58342/0 (socket says 172.21.15.49:58342) 2024-04-15T15:18:51.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996f400640 1 -- 172.21.15.49:0/1002145331 learned_addr learned my addr 172.21.15.49:0/1002145331 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:51.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996e000640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99700d1c90 0x7f99700ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996ea00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99700cf110 0x7f99701e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996f400640 1 -- 172.21.15.49:0/1002145331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99700d1c90 msgr2=0x7f99700ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996f400640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99700d1c90 0x7f99700ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996f400640 1 -- 172.21.15.49:0/1002145331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99700cf110 msgr2=0x7f99701e82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996f400640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99700cf110 0x7f99701e82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996f400640 1 -- 172.21.15.49:0/1002145331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99701e8b00 con 0x7f9970162020 2024-04-15T15:18:51.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.178+0000 7f996ea00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99700cf110 0x7f99701e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:51.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.179+0000 7f996f400640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9970162020 0x7f99700cac10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f996800cc00 tx=0x7f996800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:51.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.179+0000 7f9966a00640 1 -- 172.21.15.49:0/1002145331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996800ce10 con 0x7f9970162020 2024-04-15T15:18:51.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.179+0000 7f9966a00640 1 -- 172.21.15.49:0/1002145331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9968003990 con 0x7f9970162020 2024-04-15T15:18:51.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.179+0000 7f9966a00640 1 -- 172.21.15.49:0/1002145331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968003d60 con 0x7f9970162020 2024-04-15T15:18:51.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.179+0000 7f996fe00640 1 -- 172.21.15.49:0/1002145331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99701e8de0 con 0x7f9970162020 2024-04-15T15:18:51.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.179+0000 7f996fe00640 1 -- 172.21.15.49:0/1002145331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99701e93a0 con 0x7f9970162020 2024-04-15T15:18:51.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.180+0000 7f996fe00640 1 -- 172.21.15.49:0/1002145331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99700755c0 con 0x7f9970162020 2024-04-15T15:18:51.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.181+0000 7f9966a00640 1 -- 172.21.15.49:0/1002145331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9968003f00 con 0x7f9970162020 2024-04-15T15:18:51.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.182+0000 7f9966a00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f994c06c8f0 0x7f994c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.183+0000 7f9966a00640 1 -- 172.21.15.49:0/1002145331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f996808c070 con 0x7f9970162020 2024-04-15T15:18:51.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.183+0000 7f996ea00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f994c06c8f0 0x7f994c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.183+0000 7f996ea00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f994c06c8f0 0x7f994c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9960003eb0 tx=0x7f9960009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:51.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.187+0000 7f9966a00640 1 -- 172.21.15.49:0/1002145331 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9968014030 con 0x7f9970162020 2024-04-15T15:18:51.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.445+0000 7f996fe00640 1 -- 172.21.15.49:0/1002145331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f99700b7650 con 0x7f9970162020 2024-04-15T15:18:51.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.454+0000 7f9966a00640 1 -- 172.21.15.49:0/1002145331 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174137 (secure 0 0 0) 0x7f996805a400 con 0x7f9970162020 2024-04-15T15:18:51.458 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.459+0000 7f996fe00640 1 -- 172.21.15.49:0/1002145331 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f994c06c8f0 msgr2=0x7f994c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.459+0000 7f996fe00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f994c06c8f0 0x7f994c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9960003eb0 tx=0x7f9960009040 comp rx=0 tx=0).stop 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.459+0000 7f996fe00640 1 -- 172.21.15.49:0/1002145331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9970162020 msgr2=0x7f99700cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.459+0000 7f996fe00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9970162020 0x7f99700cac10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f996800cc00 tx=0x7f996800a5c0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.460+0000 7f996fe00640 1 -- 172.21.15.49:0/1002145331 shutdown_connections 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.460+0000 7f996fe00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f994c06c8f0 0x7f994c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.460+0000 7f996fe00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99700d1c90 0x7f99700ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.460+0000 7f996fe00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9970162020 0x7f99700cac10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.460+0000 7f996fe00640 1 --2- 172.21.15.49:0/1002145331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99700cf110 0x7f99701e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.460+0000 7f996fe00640 1 -- 172.21.15.49:0/1002145331 >> 172.21.15.49:0/1002145331 conn(0x7f997006cf00 msgr2=0x7f997015faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.460+0000 7f996fe00640 1 -- 172.21.15.49:0/1002145331 shutdown_connections 2024-04-15T15:18:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.460+0000 7f996fe00640 1 -- 172.21.15.49:0/1002145331 wait complete. 2024-04-15T15:18:51.483 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2558964566},{"type":"v1","addr":"172.21.15.49:6833","nonce":2558964566}]},"active_addr":"172.21.15.49:6833/2558964566","active_change":"2024-04-15T15:18:30.753735+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":4101890799}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3610050050}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3191704604}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3614019070}]}]} 2024-04-15T15:18:51.486 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-15T15:18:51.486 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-15T15:18:51.486 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:51.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.713+0000 7fb78e000640 1 Processor -- start 2024-04-15T15:18:51.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.713+0000 7fb78e000640 1 -- start start 2024-04-15T15:18:51.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.715+0000 7fb78e000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb788162020 0x7fb788162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.715+0000 7fb78e000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb7880d1c90 0x7fb7880d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.715+0000 7fb78e000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7880d0090 con 0x7fb7880cf110 2024-04-15T15:18:51.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.715+0000 7fb78e000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb788163440 con 0x7fb788162020 2024-04-15T15:18:51.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.715+0000 7fb78e000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fb7880b7b30 con 0x7fb7880d1c90 2024-04-15T15:18:51.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.715+0000 7fb78cc00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb788162020 0x7fb788162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.715+0000 7fb787e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb7880cf110 0x7fb7880d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44032/0 (socket says 172.21.15.49:44032) 2024-04-15T15:18:51.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.715+0000 7fb787e00640 1 -- 172.21.15.49:0/202034395 learned_addr learned my addr 172.21.15.49:0/202034395 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:51.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.716+0000 7fb78d600640 1 --2- 172.21.15.49:0/202034395 >> v2:172.21.15.107:3300/0 conn(0x7fb7880d1c90 0x7fb7880d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.716+0000 7fb78cc00640 1 -- 172.21.15.49:0/202034395 >> v2:172.21.15.107:3300/0 conn(0x7fb7880d1c90 msgr2=0x7fb7880d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:51.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.716+0000 7fb78cc00640 1 --2- 172.21.15.49:0/202034395 >> v2:172.21.15.107:3300/0 conn(0x7fb7880d1c90 0x7fb7880d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.716+0000 7fb78cc00640 1 -- 172.21.15.49:0/202034395 >> v1:172.21.15.49:6789/0 conn(0x7fb7880cf110 legacy=0x7fb7880d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.716+0000 7fb78cc00640 1 -- 172.21.15.49:0/202034395 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7880cfad0 con 0x7fb788162020 2024-04-15T15:18:51.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.716+0000 7fb787400640 1 -- 172.21.15.49:0/202034395 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089348705 0 0) 0x7fb774001110 con 0x7fb7880cf110 2024-04-15T15:18:51.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.717+0000 7fb78cc00640 1 --2- 172.21.15.49:0/202034395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 0x7fb788162f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb78808d460 tx=0x7fb77c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ba0bb2e7c722d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:51.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.717+0000 7fb787400640 1 -- 172.21.15.49:0/202034395 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c035020 con 0x7fb788162020 2024-04-15T15:18:51.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.717+0000 7fb787400640 1 -- 172.21.15.49:0/202034395 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c031830 con 0x7fb788162020 2024-04-15T15:18:51.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.717+0000 7fb787400640 1 -- 172.21.15.49:0/202034395 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c031c00 con 0x7fb788162020 2024-04-15T15:18:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.718+0000 7fb78e000640 1 -- 172.21.15.49:0/202034395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 msgr2=0x7fb788162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.718+0000 7fb78e000640 1 --2- 172.21.15.49:0/202034395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 0x7fb788162f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb78808d460 tx=0x7fb77c02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.718+0000 7fb78e000640 1 -- 172.21.15.49:0/202034395 shutdown_connections 2024-04-15T15:18:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.718+0000 7fb78e000640 1 --2- 172.21.15.49:0/202034395 >> v2:172.21.15.107:3300/0 conn(0x7fb7880d1c90 0x7fb7880d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.718+0000 7fb78e000640 1 --2- 172.21.15.49:0/202034395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 0x7fb788162f00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.718+0000 7fb78e000640 1 -- 172.21.15.49:0/202034395 >> 172.21.15.49:0/202034395 conn(0x7fb78806cf00 msgr2=0x7fb78806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.718+0000 7fb78e000640 1 -- 172.21.15.49:0/202034395 shutdown_connections 2024-04-15T15:18:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.718+0000 7fb78e000640 1 -- 172.21.15.49:0/202034395 wait complete. 2024-04-15T15:18:51.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.720+0000 7fb78e000640 1 Processor -- start 2024-04-15T15:18:51.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.720+0000 7fb78e000640 1 -- start start 2024-04-15T15:18:51.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.721+0000 7fb78e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7880cf110 0x7fb7880c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.721+0000 7fb78e000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7880d1c90 0x7fb788160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.721+0000 7fb78e000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 0x7fb7880b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.721+0000 7fb78e000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb7880d2b90 con 0x7fb788162020 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.721+0000 7fb78e000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7880cf880 con 0x7fb7880cf110 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.721+0000 7fb78e000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb7880d2e70 con 0x7fb7880d1c90 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.721+0000 7fb78d600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 0x7fb7880b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.721+0000 7fb78d600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 0x7fb7880b8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58360/0 (socket says 172.21.15.49:58360) 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.721+0000 7fb78d600640 1 -- 172.21.15.49:0/1229220484 learned_addr learned my addr 172.21.15.49:0/1229220484 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb787e00640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7880d1c90 0x7fb788160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb78d600640 1 -- 172.21.15.49:0/1229220484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7880d1c90 msgr2=0x7fb788160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb78d600640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7880d1c90 0x7fb788160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb78cc00640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7880cf110 0x7fb7880c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb78d600640 1 -- 172.21.15.49:0/1229220484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7880cf110 msgr2=0x7fb7880c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb78d600640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7880cf110 0x7fb7880c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb78d600640 1 -- 172.21.15.49:0/1229220484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78815f5c0 con 0x7fb788162020 2024-04-15T15:18:51.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb78d600640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 0x7fb7880b8e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb780007e60 tx=0x7fb78000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:51.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb784c00640 1 -- 172.21.15.49:0/1229220484 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780014620 con 0x7fb788162020 2024-04-15T15:18:51.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb784c00640 1 -- 172.21.15.49:0/1229220484 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb780019070 con 0x7fb788162020 2024-04-15T15:18:51.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb784c00640 1 -- 172.21.15.49:0/1229220484 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780014a00 con 0x7fb788162020 2024-04-15T15:18:51.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.722+0000 7fb78e000640 1 -- 172.21.15.49:0/1229220484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78815f8a0 con 0x7fb788162020 2024-04-15T15:18:51.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.723+0000 7fb78e000640 1 -- 172.21.15.49:0/1229220484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78815fe60 con 0x7fb788162020 2024-04-15T15:18:51.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.724+0000 7fb78e000640 1 -- 172.21.15.49:0/1229220484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7880755c0 con 0x7fb788162020 2024-04-15T15:18:51.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.725+0000 7fb784c00640 1 -- 172.21.15.49:0/1229220484 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb780014ba0 con 0x7fb788162020 2024-04-15T15:18:51.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.726+0000 7fb784c00640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb76806c8f0 0x7fb76806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:51.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.726+0000 7fb784c00640 1 -- 172.21.15.49:0/1229220484 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb78008c510 con 0x7fb788162020 2024-04-15T15:18:51.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.726+0000 7fb78cc00640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb76806c8f0 0x7fb76806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:51.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.727+0000 7fb78cc00640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb76806c8f0 0x7fb76806edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb77c036000 tx=0x7fb77c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:51.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.731+0000 7fb784c00640 1 -- 172.21.15.49:0/1229220484 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb78005a8a0 con 0x7fb788162020 2024-04-15T15:18:51.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.944+0000 7fb78e000640 1 -- 172.21.15.49:0/1229220484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7880b7650 con 0x7fb788162020 2024-04-15T15:18:51.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.946+0000 7fb784c00640 1 -- 172.21.15.49:0/1229220484 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7fb78005a1a0 con 0x7fb788162020 2024-04-15T15:18:51.948 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:51.948 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 -- 172.21.15.49:0/1229220484 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb76806c8f0 msgr2=0x7fb76806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb76806c8f0 0x7fb76806edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb77c036000 tx=0x7fb77c008030 comp rx=0 tx=0).stop 2024-04-15T15:18:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 -- 172.21.15.49:0/1229220484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 msgr2=0x7fb7880b8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 0x7fb7880b8e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb780007e60 tx=0x7fb78000e4b0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 -- 172.21.15.49:0/1229220484 shutdown_connections 2024-04-15T15:18:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb76806c8f0 0x7fb76806edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7880d1c90 0x7fb788160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb788162020 0x7fb7880b8e30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 --2- 172.21.15.49:0/1229220484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7880cf110 0x7fb7880c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 -- 172.21.15.49:0/1229220484 >> 172.21.15.49:0/1229220484 conn(0x7fb78806cf00 msgr2=0x7fb78806d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 -- 172.21.15.49:0/1229220484 shutdown_connections 2024-04-15T15:18:51.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:51.949+0000 7fb78e000640 1 -- 172.21.15.49:0/1229220484 wait complete. 2024-04-15T15:18:51.970 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-15T15:18:51.970 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:52.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.195+0000 7f2521000640 1 Processor -- start 2024-04-15T15:18:52.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.196+0000 7f2521000640 1 -- start start 2024-04-15T15:18:52.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.197+0000 7f2521000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f251c0d1c90 0x7f251c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.197+0000 7f2521000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 0x7f251c0d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.197+0000 7f2521000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f251c0d0090 con 0x7f251c162020 2024-04-15T15:18:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.197+0000 7f2521000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f251c163440 con 0x7f251c0d1c90 2024-04-15T15:18:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.197+0000 7f2521000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f251c0b7b30 con 0x7f251c0cf110 2024-04-15T15:18:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.198+0000 7f251aa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f251c0d1c90 0x7f251c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.198+0000 7f251aa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f251c0d1c90 0x7f251c162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58368/0 (socket says 172.21.15.49:58368) 2024-04-15T15:18:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.198+0000 7f251aa00640 1 -- 172.21.15.49:0/1474948602 learned_addr learned my addr 172.21.15.49:0/1474948602 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.198+0000 7f251be00640 1 --2- 172.21.15.49:0/1474948602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 0x7f251c0d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.198+0000 7f251aa00640 1 -- 172.21.15.49:0/1474948602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 msgr2=0x7f251c0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.198+0000 7f251aa00640 1 --2- 172.21.15.49:0/1474948602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 0x7f251c0d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.198+0000 7f251aa00640 1 -- 172.21.15.49:0/1474948602 >> v1:172.21.15.107:6789/0 conn(0x7f251c162020 legacy=0x7f251c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.198+0000 7f251aa00640 1 -- 172.21.15.49:0/1474948602 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c0cfad0 con 0x7f251c0d1c90 2024-04-15T15:18:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.200+0000 7f251aa00640 1 --2- 172.21.15.49:0/1474948602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 0x7f251c162f00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2508009870 tx=0x7f250802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2f7022361977a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.200+0000 7f251a000640 1 -- 172.21.15.49:0/1474948602 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508034070 con 0x7f251c0d1c90 2024-04-15T15:18:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.200+0000 7f251a000640 1 -- 172.21.15.49:0/1474948602 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2508030840 con 0x7f251c0d1c90 2024-04-15T15:18:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.200+0000 7f251a000640 1 -- 172.21.15.49:0/1474948602 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508030c10 con 0x7f251c0d1c90 2024-04-15T15:18:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.201+0000 7f2521000640 1 -- 172.21.15.49:0/1474948602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 msgr2=0x7f251c162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.201+0000 7f2521000640 1 --2- 172.21.15.49:0/1474948602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 0x7f251c162f00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2508009870 tx=0x7f250802b980 comp rx=0 tx=0).stop 2024-04-15T15:18:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.201+0000 7f2521000640 1 -- 172.21.15.49:0/1474948602 shutdown_connections 2024-04-15T15:18:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.201+0000 7f2521000640 1 --2- 172.21.15.49:0/1474948602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 0x7f251c0d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.201+0000 7f2521000640 1 --2- 172.21.15.49:0/1474948602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 0x7f251c162f00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.201+0000 7f2521000640 1 -- 172.21.15.49:0/1474948602 >> 172.21.15.49:0/1474948602 conn(0x7f251c06cf00 msgr2=0x7f251c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.201+0000 7f2521000640 1 -- 172.21.15.49:0/1474948602 shutdown_connections 2024-04-15T15:18:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.201+0000 7f2521000640 1 -- 172.21.15.49:0/1474948602 wait complete. 2024-04-15T15:18:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.203+0000 7f2521000640 1 Processor -- start 2024-04-15T15:18:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.203+0000 7f2521000640 1 -- start start 2024-04-15T15:18:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f2521000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 0x7f251c0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f2521000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 0x7f251c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f2521000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f251c162020 0x7f251c154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f2521000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f251c0d2b90 con 0x7f251c0d1c90 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f2521000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f251c0cf880 con 0x7f251c162020 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f2521000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f251c0d2e70 con 0x7f251c0cf110 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f251aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 0x7f251c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f251aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 0x7f251c1548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58372/0 (socket says 172.21.15.49:58372) 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f251aa00640 1 -- 172.21.15.49:0/2890698917 learned_addr learned my addr 172.21.15.49:0/2890698917 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f251be00640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f251c162020 0x7f251c154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f251b400640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 0x7f251c0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f251aa00640 1 -- 172.21.15.49:0/2890698917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 msgr2=0x7f251c0ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f251aa00640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 0x7f251c0ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f251aa00640 1 -- 172.21.15.49:0/2890698917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f251c162020 msgr2=0x7f251c154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.204+0000 7f251aa00640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f251c162020 0x7f251c154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.205+0000 7f251aa00640 1 -- 172.21.15.49:0/2890698917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c1f5090 con 0x7f251c0d1c90 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.205+0000 7f251be00640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f251c162020 0x7f251c154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.205+0000 7f251b400640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 0x7f251c0ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.205+0000 7f251aa00640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 0x7f251c1548b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2508000c00 tx=0x7f2508004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.205+0000 7f2513400640 1 -- 172.21.15.49:0/2890698917 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508048070 con 0x7f251c0d1c90 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.205+0000 7f2513400640 1 -- 172.21.15.49:0/2890698917 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2508030640 con 0x7f251c0d1c90 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.205+0000 7f2521000640 1 -- 172.21.15.49:0/2890698917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c1f5370 con 0x7f251c0d1c90 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.205+0000 7f2513400640 1 -- 172.21.15.49:0/2890698917 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508034040 con 0x7f251c0d1c90 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.206+0000 7f2521000640 1 -- 172.21.15.49:0/2890698917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1f5930 con 0x7f251c0d1c90 2024-04-15T15:18:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.207+0000 7f2521000640 1 -- 172.21.15.49:0/2890698917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f251c0755c0 con 0x7f251c0d1c90 2024-04-15T15:18:52.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.208+0000 7f2513400640 1 -- 172.21.15.49:0/2890698917 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f250804b070 con 0x7f251c0d1c90 2024-04-15T15:18:52.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.209+0000 7f2513400640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24fc06c8f0 0x7f24fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.209+0000 7f2513400640 1 -- 172.21.15.49:0/2890698917 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f25080adaa0 con 0x7f251c0d1c90 2024-04-15T15:18:52.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.209+0000 7f251b400640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24fc06c8f0 0x7f24fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.210+0000 7f251b400640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24fc06c8f0 0x7f24fc06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f250c001680 tx=0x7f250c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.213+0000 7f2513400640 1 -- 172.21.15.49:0/2890698917 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f250807bff0 con 0x7f251c0d1c90 2024-04-15T15:18:52.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.423+0000 7f2521000640 1 -- 172.21.15.49:0/2890698917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f251c0b7650 con 0x7f251c0d1c90 2024-04-15T15:18:52.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.425+0000 7f2513400640 1 -- 172.21.15.49:0/2890698917 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7f2508030810 con 0x7f251c0d1c90 2024-04-15T15:18:52.427 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:52.427 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:52.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.427+0000 7f2521000640 1 -- 172.21.15.49:0/2890698917 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24fc06c8f0 msgr2=0x7f24fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24fc06c8f0 0x7f24fc06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f250c001680 tx=0x7f250c009040 comp rx=0 tx=0).stop 2024-04-15T15:18:52.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 -- 172.21.15.49:0/2890698917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 msgr2=0x7f251c1548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 0x7f251c1548b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2508000c00 tx=0x7f2508004730 comp rx=0 tx=0).stop 2024-04-15T15:18:52.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 -- 172.21.15.49:0/2890698917 shutdown_connections 2024-04-15T15:18:52.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f24fc06c8f0 0x7f24fc06edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f251c162020 0x7f251c154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251c0d1c90 0x7f251c1548b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 --2- 172.21.15.49:0/2890698917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f251c0cf110 0x7f251c0ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 -- 172.21.15.49:0/2890698917 >> 172.21.15.49:0/2890698917 conn(0x7f251c06cf00 msgr2=0x7f251c15faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 -- 172.21.15.49:0/2890698917 shutdown_connections 2024-04-15T15:18:52.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.428+0000 7f2521000640 1 -- 172.21.15.49:0/2890698917 wait complete. 2024-04-15T15:18:52.451 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-15T15:18:52.451 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-15T15:18:52.452 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-15T15:18:52.452 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-15T15:18:52.452 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-15T15:18:52.453 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-15T15:18:52.453 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-15T15:18:52.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-15T15:18:52.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.775+0000 7fac5e000640 1 Processor -- start 2024-04-15T15:18:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.775+0000 7fac5e000640 1 -- start start 2024-04-15T15:18:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.776+0000 7f2d2ca00640 1 Processor -- start 2024-04-15T15:18:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.776+0000 7f2d2ca00640 1 -- start start 2024-04-15T15:18:52.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.777+0000 7fac5e000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fac5000c0a0 0x7fac50038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.777+0000 7fac5e000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac50033ef0 0x7fac5004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.777+0000 7fac5e000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac500544c0 con 0x7fac50053c70 2024-04-15T15:18:52.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.777+0000 7fac5e000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fac500541e0 con 0x7fac5000c0a0 2024-04-15T15:18:52.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.777+0000 7fac5e000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fac50005f30 con 0x7fac50033ef0 2024-04-15T15:18:52.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.778+0000 7fac5d600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac50033ef0 0x7fac5004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.778+0000 7fac5d600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac50033ef0 0x7fac5004fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:45938/0 (socket says 172.21.15.49:45938) 2024-04-15T15:18:52.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.778+0000 7fac5d600640 1 -- 172.21.15.49:0/3221972515 learned_addr learned my addr 172.21.15.49:0/3221972515 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.777+0000 7fc6f5a00640 1 Processor -- start 2024-04-15T15:18:52.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.777+0000 7fc6f5a00640 1 -- start start 2024-04-15T15:18:52.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7f2d2ca00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2d280d0070 0x7f2d2815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7f2d2ca00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2d280d0450 0x7f2d280723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.780+0000 7f2d25400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2d280d0450 0x7f2d280723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.780+0000 7f2d25400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2d280d0450 0x7f2d280723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:50278/0 (socket says 172.21.15.49:50278) 2024-04-15T15:18:52.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.780+0000 7f2d25e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2d280d0070 0x7f2d2815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.781+0000 7f2d25400640 1 -- 172.21.15.49:0/2526261408 learned_addr learned my addr 172.21.15.49:0/2526261408 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7fc6f5a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc6f00cf620 0x7fc6f01633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7fc6f5a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc6f01669d0 0x7fc6f00d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7fc6f5a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6f00d1170 con 0x7fc6f0163f90 2024-04-15T15:18:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7fc6f5a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f0167900 con 0x7fc6f00cf620 2024-04-15T15:18:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7fc6f5a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f0164370 con 0x7fc6f01669d0 2024-04-15T15:18:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.780+0000 7fc6eea00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc6f0163f90 0x7fc6f00cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44036/0 (socket says 172.21.15.49:44036) 2024-04-15T15:18:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.780+0000 7fc6eea00640 1 -- 172.21.15.49:0/3152187714 learned_addr learned my addr 172.21.15.49:0/3152187714 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.780+0000 7fc6ef400640 1 --2- 172.21.15.49:0/3152187714 >> v2:172.21.15.107:3300/0 conn(0x7fc6f01669d0 0x7fc6f00d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.781+0000 7fc6ee000640 1 --2- 172.21.15.49:0/3152187714 >> v2:172.21.15.49:3300/0 conn(0x7fc6f00cf620 0x7fc6f01633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7fac5d600640 1 -- 172.21.15.49:0/3221972515 >> v2:172.21.15.49:3300/0 conn(0x7fac5000c0a0 msgr2=0x7fac50038080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7fac5d600640 1 --2- 172.21.15.49:0/3221972515 >> v2:172.21.15.49:3300/0 conn(0x7fac5000c0a0 0x7fac50038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7fac5d600640 1 -- 172.21.15.49:0/3221972515 >> v1:172.21.15.107:6789/0 conn(0x7fac50053c70 legacy=0x7fac500df220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-15T15:18:52.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.779+0000 7fac5d600640 1 -- 172.21.15.49:0/3221972515 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac50006210 con 0x7fac50033ef0 2024-04-15T15:18:52.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.781+0000 7f2d2ca00640 1 -- 172.21.15.49:0/2526261408 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2806e490 con 0x7f2d280b56e0 2024-04-15T15:18:52.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.781+0000 7fac5d600640 1 --2- 172.21.15.49:0/3221972515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac50033ef0 0x7fac5004fa90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fac44009870 tx=0x7fac4402b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338dc71c73dfdbeb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.782+0000 7f2d2ca00640 1 -- 172.21.15.49:0/2526261408 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2815bf10 con 0x7f2d280d0070 2024-04-15T15:18:52.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.781+0000 7fc6ef400640 1 -- 172.21.15.49:0/3152187714 >> v2:172.21.15.49:3300/0 conn(0x7fc6f00cf620 msgr2=0x7fc6f01633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.781+0000 7fc6ef400640 1 --2- 172.21.15.49:0/3152187714 >> v2:172.21.15.49:3300/0 conn(0x7fc6f00cf620 0x7fc6f01633e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.781+0000 7fc6ef400640 1 -- 172.21.15.49:0/3152187714 >> v1:172.21.15.49:6789/0 conn(0x7fc6f0163f90 legacy=0x7fc6f00cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.782+0000 7f2d2ca00640 1 -- 172.21.15.49:0/2526261408 --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2806e770 con 0x7f2d280d0450 2024-04-15T15:18:52.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.782+0000 7fac57400640 1 -- 172.21.15.49:0/3221972515 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4402e070 con 0x7fac50033ef0 2024-04-15T15:18:52.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.782+0000 7fac57400640 1 -- 172.21.15.49:0/3221972515 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac44037070 con 0x7fac50033ef0 2024-04-15T15:18:52.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.782+0000 7fac57400640 1 -- 172.21.15.49:0/3221972515 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44031a40 con 0x7fac50033ef0 2024-04-15T15:18:52.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.783+0000 7f2d25400640 1 -- 172.21.15.49:0/2526261408 >> v2:172.21.15.49:3300/0 conn(0x7f2d280d0070 msgr2=0x7f2d2815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.783+0000 7f2d25400640 1 --2- 172.21.15.49:0/2526261408 >> v2:172.21.15.49:3300/0 conn(0x7f2d280d0070 0x7f2d2815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.783+0000 7f2d25400640 1 -- 172.21.15.49:0/2526261408 >> v1:172.21.15.49:6789/0 conn(0x7f2d280b56e0 legacy=0x7f2d2815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.783+0000 7fac5e000640 1 -- 172.21.15.49:0/3221972515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac50033ef0 msgr2=0x7fac5004fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.783+0000 7fac5e000640 1 --2- 172.21.15.49:0/3221972515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac50033ef0 0x7fac5004fa90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fac44009870 tx=0x7fac4402b880 comp rx=0 tx=0).stop 2024-04-15T15:18:52.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.782+0000 7fc6ef400640 1 -- 172.21.15.49:0/3152187714 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f0167060 con 0x7fc6f01669d0 2024-04-15T15:18:52.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.783+0000 7f2d25400640 1 -- 172.21.15.49:0/2526261408 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2806ea50 con 0x7f2d280d0450 2024-04-15T15:18:52.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.784+0000 7fac5e000640 1 -- 172.21.15.49:0/3221972515 shutdown_connections 2024-04-15T15:18:52.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.784+0000 7fac5e000640 1 --2- 172.21.15.49:0/3221972515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac50033ef0 0x7fac5004fa90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.784+0000 7fac5e000640 1 --2- 172.21.15.49:0/3221972515 >> v2:172.21.15.49:3300/0 conn(0x7fac5000c0a0 0x7fac50038080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.784+0000 7fac5e000640 1 -- 172.21.15.49:0/3221972515 >> 172.21.15.49:0/3221972515 conn(0x7fac50009fb0 msgr2=0x7fac5000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.784+0000 7fb829800640 1 Processor -- start 2024-04-15T15:18:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.784+0000 7fc6ef400640 1 --2- 172.21.15.49:0/3152187714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f01669d0 0x7fc6f00d2b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc6f008dcf0 tx=0x7fc6e002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e4a3e52df5c365e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.785+0000 7fac5e000640 1 -- 172.21.15.49:0/3221972515 shutdown_connections 2024-04-15T15:18:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.785+0000 7fac5e000640 1 -- 172.21.15.49:0/3221972515 wait complete. 2024-04-15T15:18:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.785+0000 7f2d25400640 1 --2- 172.21.15.49:0/2526261408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d280d0450 0x7f2d280723b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2d2806f660 tx=0x7f2d1402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57206c0846a8f3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.785+0000 7fc6ed600640 1 -- 172.21.15.49:0/3152187714 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0035020 con 0x7fc6f01669d0 2024-04-15T15:18:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.785+0000 7fc6ed600640 1 -- 172.21.15.49:0/3152187714 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e0031830 con 0x7fc6f01669d0 2024-04-15T15:18:52.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.785+0000 7fc6ed600640 1 -- 172.21.15.49:0/3152187714 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0031c60 con 0x7fc6f01669d0 2024-04-15T15:18:52.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.785+0000 7fb829800640 1 -- start start 2024-04-15T15:18:52.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.786+0000 7fc6f5a00640 1 -- 172.21.15.49:0/3152187714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f01669d0 msgr2=0x7fc6f00d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.786+0000 7fc6f5a00640 1 --2- 172.21.15.49:0/3152187714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f01669d0 0x7fc6f00d2b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc6f008dcf0 tx=0x7fc6e002b8f0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.786+0000 7fc6f5a00640 1 -- 172.21.15.49:0/3152187714 shutdown_connections 2024-04-15T15:18:52.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.786+0000 7fc6f5a00640 1 --2- 172.21.15.49:0/3152187714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f01669d0 0x7fc6f00d2b50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.786+0000 7fc6f5a00640 1 --2- 172.21.15.49:0/3152187714 >> v2:172.21.15.49:3300/0 conn(0x7fc6f00cf620 0x7fc6f01633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.786+0000 7fc6f5a00640 1 -- 172.21.15.49:0/3152187714 >> 172.21.15.49:0/3152187714 conn(0x7fc6f00d1970 msgr2=0x7fc6f00d48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.786+0000 7fc6f5a00640 1 -- 172.21.15.49:0/3152187714 shutdown_connections 2024-04-15T15:18:52.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.786+0000 7fc6f5a00640 1 -- 172.21.15.49:0/3152187714 wait complete. 2024-04-15T15:18:52.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.787+0000 7f2d24a00640 1 -- 172.21.15.49:0/2526261408 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14007ce0 con 0x7f2d280d0450 2024-04-15T15:18:52.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.787+0000 7fb829800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8240b3e00 0x7fb8240b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.787+0000 7fb829800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb8240b5a20 0x7fb8240b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.787+0000 7fb829800640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8240cf270 con 0x7fb824072950 2024-04-15T15:18:52.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.787+0000 7fb829800640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb824158900 con 0x7fb8240b5a20 2024-04-15T15:18:52.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.787+0000 7fb829800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb8240d0350 con 0x7fb8240b3e00 2024-04-15T15:18:52.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.787+0000 7f2d24a00640 1 -- 172.21.15.49:0/2526261408 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d14036070 con 0x7f2d280d0450 2024-04-15T15:18:52.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.787+0000 7fac5e000640 1 Processor -- start 2024-04-15T15:18:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.787+0000 7fb823e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb824072950 0x7fb8240ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44060/0 (socket says 172.21.15.49:44060) 2024-04-15T15:18:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.788+0000 7fb823e00640 1 -- 172.21.15.49:0/1188024899 learned_addr learned my addr 172.21.15.49:0/1188024899 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.788+0000 7fb823400640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8240b3e00 0x7fb8240b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.787+0000 7f2d24a00640 1 -- 172.21.15.49:0/2526261408 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140092f0 con 0x7f2d280d0450 2024-04-15T15:18:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.788+0000 7f2d2ca00640 1 -- 172.21.15.49:0/2526261408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d280d0450 msgr2=0x7f2d280723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.788+0000 7f2d2ca00640 1 --2- 172.21.15.49:0/2526261408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d280d0450 0x7f2d280723b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2d2806f660 tx=0x7f2d1402b910 comp rx=0 tx=0).stop 2024-04-15T15:18:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.789+0000 7fac5e000640 1 -- start start 2024-04-15T15:18:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.789+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2724343097 0 0) 0x7fb8240cf270 con 0x7fb824072950 2024-04-15T15:18:52.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.789+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb808003520 con 0x7fb824072950 2024-04-15T15:18:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.789+0000 7fc6f5a00640 1 Processor -- start 2024-04-15T15:18:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.789+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3535884464 0 0) 0x7fb808003520 con 0x7fb824072950 2024-04-15T15:18:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.789+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8240b3e00 msgr2=0x7fb8240b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.789+0000 7fb822a00640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8240b3e00 0x7fb8240b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.789+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 >> v2:172.21.15.49:3300/0 conn(0x7fb8240b5a20 msgr2=0x7fb8240b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7f2d2ca00640 1 -- 172.21.15.49:0/2526261408 shutdown_connections 2024-04-15T15:18:52.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7f2d2ca00640 1 --2- 172.21.15.49:0/2526261408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d280d0450 0x7f2d280723b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7f2d2ca00640 1 --2- 172.21.15.49:0/2526261408 >> v2:172.21.15.49:3300/0 conn(0x7f2d280d0070 0x7f2d2815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7f2d2ca00640 1 -- 172.21.15.49:0/2526261408 >> 172.21.15.49:0/2526261408 conn(0x7f2d2806b930 msgr2=0x7f2d2806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7fb822a00640 1 --2- 172.21.15.49:0/1188024899 >> v2:172.21.15.49:3300/0 conn(0x7fb8240b5a20 0x7fb8240b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8240cf550 con 0x7fb824072950 2024-04-15T15:18:52.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fb80c001690 con 0x7fb824072950 2024-04-15T15:18:52.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 >> v1:172.21.15.49:6789/0 conn(0x7fb824072950 legacy=0x7fb8240ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7fb822a00640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8080039b0 0x7fb808003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7fb822a00640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8080043a0 0x7fb808004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7f2d2ca00640 1 -- 172.21.15.49:0/2526261408 shutdown_connections 2024-04-15T15:18:52.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.790+0000 7fac5e000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac5000c0a0 0x7fac5004e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7f2d2ca00640 1 -- 172.21.15.49:0/2526261408 wait complete. 2024-04-15T15:18:52.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.789+0000 7fc6f5a00640 1 -- start start 2024-04-15T15:18:52.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fb822a00640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8240b3e00 0x7fb8080050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fc6f5a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f00cf620 0x7fc6f006eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fc6f5a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6f0163f90 0x7fc6f00ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fc6f5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f01669d0 0x7fc6f01fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fac5e000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac50033ef0 0x7fac5004e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fc6f5a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f0167620 con 0x7fc6f00cf620 2024-04-15T15:18:52.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fc6f5a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f00d26c0 con 0x7fc6f01669d0 2024-04-15T15:18:52.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fc6f5a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f00d3090 con 0x7fc6f0163f90 2024-04-15T15:18:52.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb824158900 con 0x7fb8080043a0 2024-04-15T15:18:52.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fac5e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac50053c70 0x7fac50049770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fb823e00640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8080039b0 0x7fb808003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8240d0350 con 0x7fb8080039b0 2024-04-15T15:18:52.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb80800bb00 con 0x7fb8240b3e00 2024-04-15T15:18:52.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fb823400640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8080043a0 0x7fb808004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fac5e000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac50037a90 con 0x7fac50033ef0 2024-04-15T15:18:52.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fac5e000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac50037910 con 0x7fac50053c70 2024-04-15T15:18:52.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.791+0000 7fac5e000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fac5004ffd0 con 0x7fac5000c0a0 2024-04-15T15:18:52.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fac5cc00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac5000c0a0 0x7fac5004e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fac5cc00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac5000c0a0 0x7fac5004e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:45948/0 (socket says 172.21.15.49:45948) 2024-04-15T15:18:52.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fac5cc00640 1 -- 172.21.15.49:0/1810286613 learned_addr learned my addr 172.21.15.49:0/1810286613 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fc6ef400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f01669d0 0x7fc6f01fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fc6ef400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f01669d0 0x7fc6f01fbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:50306/0 (socket says 172.21.15.49:50306) 2024-04-15T15:18:52.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fc6ef400640 1 -- 172.21.15.49:0/2649509414 learned_addr learned my addr 172.21.15.49:0/2649509414 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fac57e00640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac50033ef0 0x7fac5004e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fc6eea00640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f00cf620 0x7fc6f006eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fb823400640 1 -- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8240b3e00 msgr2=0x7fb8080050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fb823400640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8240b3e00 0x7fb8080050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fac5cc00640 1 -- 172.21.15.49:0/1810286613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac50053c70 msgr2=0x7fac50049770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fc6ee000640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6f0163f90 0x7fc6f00ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fac5cc00640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac50053c70 0x7fac50049770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fac5cc00640 1 -- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac50033ef0 msgr2=0x7fac5004e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.792+0000 7fb823400640 1 -- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8080039b0 msgr2=0x7fb808003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fb823400640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8080039b0 0x7fb808003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fb823400640 1 -- 172.21.15.49:0/1188024899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8240cf550 con 0x7fb8080043a0 2024-04-15T15:18:52.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fac5cc00640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac50033ef0 0x7fac5004e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fac5cc00640 1 -- 172.21.15.49:0/1810286613 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac50045700 con 0x7fac5000c0a0 2024-04-15T15:18:52.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fb823e00640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8080039b0 0x7fb808003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:18:52.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fc6ef400640 1 -- 172.21.15.49:0/2649509414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6f0163f90 msgr2=0x7fc6f00ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fc6ef400640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6f0163f90 0x7fc6f00ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.793+0000 7fc6ef400640 1 -- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f00cf620 msgr2=0x7fc6f006eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.794+0000 7fc6ef400640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f00cf620 0x7fc6f006eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.794+0000 7fc6ef400640 1 -- 172.21.15.49:0/2649509414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f00720f0 con 0x7fc6f01669d0 2024-04-15T15:18:52.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.794+0000 7fc6eea00640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f00cf620 0x7fc6f006eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:18:52.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.794+0000 7fac5cc00640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac5000c0a0 0x7fac5004e1d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00b500 tx=0x7fac4c00b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.794+0000 7fc6ef400640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f01669d0 0x7fc6f01fbe60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009890 tx=0x7fc6e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.795+0000 7fac54c00640 1 -- 172.21.15.49:0/1810286613 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c014070 con 0x7fac5000c0a0 2024-04-15T15:18:52.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.795+0000 7fac54c00640 1 -- 172.21.15.49:0/1810286613 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac4c010870 con 0x7fac5000c0a0 2024-04-15T15:18:52.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.795+0000 7fac54c00640 1 -- 172.21.15.49:0/1810286613 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c010c60 con 0x7fac5000c0a0 2024-04-15T15:18:52.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.795+0000 7fc6e6a00640 1 -- 172.21.15.49:0/2649509414 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0004450 con 0x7fc6f01669d0 2024-04-15T15:18:52.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.795+0000 7fc6e6a00640 1 -- 172.21.15.49:0/2649509414 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e0003610 con 0x7fc6f01669d0 2024-04-15T15:18:52.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.795+0000 7fc6e6a00640 1 -- 172.21.15.49:0/2649509414 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0003a40 con 0x7fc6f01669d0 2024-04-15T15:18:52.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.795+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f0071850 con 0x7fc6f01669d0 2024-04-15T15:18:52.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.795+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac500459e0 con 0x7fac5000c0a0 2024-04-15T15:18:52.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.795+0000 7fb823400640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8080043a0 0x7fb808004880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb814009ac0 tx=0x7fb808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b172fd0123fd17 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.796+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f0071e10 con 0x7fc6f01669d0 2024-04-15T15:18:52.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.796+0000 7f2d2ca00640 1 Processor -- start 2024-04-15T15:18:52.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.796+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fac50045fa0 con 0x7fac5000c0a0 2024-04-15T15:18:52.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.796+0000 7f2d2ca00640 1 -- start start 2024-04-15T15:18:52.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.796+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814039070 con 0x7fb8080043a0 2024-04-15T15:18:52.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.797+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb81402c710 con 0x7fb8080043a0 2024-04-15T15:18:52.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.797+0000 7fb822a00640 1 -- 172.21.15.49:0/1188024899 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814039070 con 0x7fb8080043a0 2024-04-15T15:18:52.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.797+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac50005390 con 0x7fac5000c0a0 2024-04-15T15:18:52.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7fb829800640 1 -- 172.21.15.49:0/1188024899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8080043a0 msgr2=0x7fb808004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7fb829800640 1 --2- 172.21.15.49:0/1188024899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8080043a0 0x7fb808004880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb814009ac0 tx=0x7fb808005830 comp rx=0 tx=0).stop 2024-04-15T15:18:52.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6f00c9b60 con 0x7fc6f01669d0 2024-04-15T15:18:52.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7faffd400640 1 Processor -- start 2024-04-15T15:18:52.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7fb828e00640 1 -- 172.21.15.49:0/1188024899 reap_dead start 2024-04-15T15:18:52.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7f2d2ca00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d280b56e0 0x7f2d2813fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7f2d2ca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d280d0070 0x7f2d28144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7fb829800640 1 -- 172.21.15.49:0/1188024899 shutdown_connections 2024-04-15T15:18:52.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7fb829800640 1 -- 172.21.15.49:0/1188024899 >> 172.21.15.49:0/1188024899 conn(0x7fb824071e30 msgr2=0x7fb824072240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7f2d2ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d280d0450 0x7f2d28146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7f2d2ca00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d280b4a50 con 0x7f2d280d0070 2024-04-15T15:18:52.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7f2d2ca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2806f010 con 0x7f2d280d0450 2024-04-15T15:18:52.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7f2d2ca00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f2d280ceb70 con 0x7f2d280b56e0 2024-04-15T15:18:52.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.798+0000 7fb829800640 1 -- 172.21.15.49:0/1188024899 shutdown_connections 2024-04-15T15:18:52.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.799+0000 7f2d26800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d280d0450 0x7f2d28146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.799+0000 7fb829800640 1 -- 172.21.15.49:0/1188024899 wait complete. 2024-04-15T15:18:52.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.799+0000 7fa9a3200640 1 Processor -- start 2024-04-15T15:18:52.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7fac54c00640 1 -- 172.21.15.49:0/1810286613 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac4c017070 con 0x7fac5000c0a0 2024-04-15T15:18:52.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.799+0000 7f2d26800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d280d0450 0x7f2d28146320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:50320/0 (socket says 172.21.15.49:50320) 2024-04-15T15:18:52.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.799+0000 7f2d26800640 1 -- 172.21.15.49:0/3699760388 learned_addr learned my addr 172.21.15.49:0/3699760388 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.799+0000 7f2d25e00640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d280b56e0 0x7f2d2813fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.800+0000 7f2d25400640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d280d0070 0x7f2d28144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.800+0000 7fc6e6a00640 1 -- 172.21.15.49:0/2649509414 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6e0038020 con 0x7fc6f01669d0 2024-04-15T15:18:52.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7fc6e6a00640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc6d006c940 0x7fc6d006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7fc6e6a00640 1 -- 172.21.15.49:0/2649509414 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc6e00ae0b0 con 0x7fc6f01669d0 2024-04-15T15:18:52.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7fc6e6a00640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] conn(0x7fc6d0074b60 0x7fc6d0076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7fc6e6a00640 1 -- 172.21.15.49:0/2649509414 --> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6e000af30 con 0x7fc6d0074b60 2024-04-15T15:18:52.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7fc6e6a00640 1 -- 172.21.15.49:0/2649509414 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc6e00ae460 con 0x7fc6f01669d0 2024-04-15T15:18:52.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7fc6eea00640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc6d006c940 0x7fc6d006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.800+0000 7fd0c4800640 1 Processor -- start 2024-04-15T15:18:52.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.800+0000 7fd0c4800640 1 -- start start 2024-04-15T15:18:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.800+0000 7fa9a3200640 1 -- start start 2024-04-15T15:18:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.799+0000 7faffd400640 1 -- start start 2024-04-15T15:18:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7faffd400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faff815ec40 0x7faff80d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7faffd400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faff80d2c30 0x7faff80ba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7faffd400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff80ba8d0 con 0x7faff80d1300 2024-04-15T15:18:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7faffd400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7faff80d3090 con 0x7faff815ec40 2024-04-15T15:18:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7faffd400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7faff80d32c0 con 0x7faff80d2c30 2024-04-15T15:18:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7faff6800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faff815ec40 0x7faff80d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7faff6800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faff815ec40 0x7faff80d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:50336/0 (socket says 172.21.15.49:50336) 2024-04-15T15:18:52.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.801+0000 7faff6800640 1 -- 172.21.15.49:0/2879622558 learned_addr learned my addr 172.21.15.49:0/2879622558 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7faff5e00640 1 --2- 172.21.15.49:0/2879622558 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faff80d2c30 0x7faff80ba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7fa9a3200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c0d4360 0x7fa99c0d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7fa9a3200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa99c0d1300 0x7fa99c0d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7fd0c4800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc15ec40 0x7fd0bc0d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7fd0c4800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd0bc0d1300 0x7fd0bc0d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7faff5400640 1 -- 172.21.15.49:0/2879622558 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397022677 0 0) 0x7faff80ba8d0 con 0x7faff80d1300 2024-04-15T15:18:52.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7faff5400640 1 -- 172.21.15.49:0/2879622558 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafdc003520 con 0x7faff80d1300 2024-04-15T15:18:52.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7f2d26800640 1 -- 172.21.15.49:0/3699760388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d280b56e0 msgr2=0x7f2d2813fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7f2d26800640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d280b56e0 0x7f2d2813fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7f2d26800640 1 -- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d280d0070 msgr2=0x7f2d28144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7f2d26800640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d280d0070 0x7f2d28144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7fb829800640 1 Processor -- start 2024-04-15T15:18:52.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7f2d26800640 1 -- 172.21.15.49:0/3699760388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d281454c0 con 0x7f2d280d0450 2024-04-15T15:18:52.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7f2d25400640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d280d0070 0x7f2d28144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:52.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fb829800640 1 -- start start 2024-04-15T15:18:52.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fa99b400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c0d4360 0x7fa99c0d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7faff6800640 1 -- 172.21.15.49:0/2879622558 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faff80d2c30 msgr2=0x7faff80ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7faff6800640 1 --2- 172.21.15.49:0/2879622558 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faff80d2c30 0x7faff80ba390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7faff6800640 1 -- 172.21.15.49:0/2879622558 >> v1:172.21.15.107:6789/0 conn(0x7faff80d1300 legacy=0x7faff80b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7faff6800640 1 -- 172.21.15.49:0/2879622558 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff80d35a0 con 0x7faff815ec40 2024-04-15T15:18:52.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fd0c2800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd0bc0d1300 0x7fd0bc0d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fd0c2800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd0bc0d1300 0x7fd0bc0d17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:50344/0 (socket says 172.21.15.49:50344) 2024-04-15T15:18:52.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fd0c2800640 1 -- 172.21.15.49:0/236849071 learned_addr learned my addr 172.21.15.49:0/236849071 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fc6eea00640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc6d006c940 0x7fc6d006ee00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8005c00 tx=0x7fc6e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fc6ee000640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] conn(0x7fc6d0074b60 0x7fc6d0076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.802+0000 7fd0c4800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0bc0d32c0 con 0x7fd0bc0d2c30 2024-04-15T15:18:52.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fd0c4800640 1 -- 172.21.15.49:0/236849071 --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fd0bc0d3090 con 0x7fd0bc0d1300 2024-04-15T15:18:52.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fd0c4800640 1 -- 172.21.15.49:0/236849071 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd0bc0d35a0 con 0x7fd0bc15ec40 2024-04-15T15:18:52.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fac54c00640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fac3806c940 0x7fac3806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fac54c00640 1 -- 172.21.15.49:0/1810286613 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fac4c08beb0 con 0x7fac5000c0a0 2024-04-15T15:18:52.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fa99b400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c0d4360 0x7fa99c0d2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46002/0 (socket says 172.21.15.49:46002) 2024-04-15T15:18:52.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fa9a3200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa99c0d32c0 con 0x7fa99c15ec40 2024-04-15T15:18:52.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fa9a3200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fa99c0d35a0 con 0x7fa99c0d1300 2024-04-15T15:18:52.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fa9a3200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c0d3880 con 0x7fa99c0d4360 2024-04-15T15:18:52.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.803+0000 7fa99b400640 1 -- 172.21.15.49:0/1988089397 learned_addr learned my addr 172.21.15.49:0/1988089397 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fd0c0a00640 1 -- 172.21.15.49:0/236849071 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2548963964 0 0) 0x7fd0bc0d32c0 con 0x7fd0bc0d2c30 2024-04-15T15:18:52.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fd0c0a00640 1 -- 172.21.15.49:0/236849071 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0a4003520 con 0x7fd0bc0d2c30 2024-04-15T15:18:52.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fd0c1e00640 1 --2- 172.21.15.49:0/236849071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc15ec40 0x7fd0bc0d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fa9a1200640 1 --2- 172.21.15.49:0/1988089397 >> v2:172.21.15.107:3300/0 conn(0x7fa99c0d1300 0x7fa99c0d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fac54c00640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] conn(0x7fac38074bc0 0x7fac38076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fac57e00640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fac3806c940 0x7fac3806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fa99b400640 1 -- 172.21.15.49:0/1988089397 >> v2:172.21.15.107:3300/0 conn(0x7fa99c0d1300 msgr2=0x7fa99c0d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fc6ee000640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] conn(0x7fc6d0074b60 0x7fc6d0076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fac54c00640 1 -- 172.21.15.49:0/1810286613 --> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac4c017e50 con 0x7fac38074bc0 2024-04-15T15:18:52.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fac54c00640 1 -- 172.21.15.49:0/1810286613 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fac4c08c260 con 0x7fac5000c0a0 2024-04-15T15:18:52.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7f2d26800640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d280d0450 0x7f2d28146320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800c3c0 tx=0x7f2d1800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fb829800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8080043a0 0x7fb8240cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.804+0000 7fa99b400640 1 --2- 172.21.15.49:0/1988089397 >> v2:172.21.15.107:3300/0 conn(0x7fa99c0d1300 0x7fa99c0d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fa99b400640 1 -- 172.21.15.49:0/1988089397 >> v1:172.21.15.107:6789/0 conn(0x7fa99c15ec40 legacy=0x7fa99c0ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fa99b400640 1 -- 172.21.15.49:0/1988089397 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c0d3b60 con 0x7fa99c0d4360 2024-04-15T15:18:52.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7faff6800640 1 --2- 172.21.15.49:0/2879622558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faff815ec40 0x7faff80d4360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009870 tx=0x7fafe802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c6efbd1a304eba server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fb829800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb824204690 0x7fb824201b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fd0c2800640 1 -- 172.21.15.49:0/236849071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc15ec40 msgr2=0x7fd0bc0d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fd0c2800640 1 --2- 172.21.15.49:0/236849071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc15ec40 0x7fd0bc0d4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fd0c2800640 1 -- 172.21.15.49:0/236849071 >> v1:172.21.15.107:6789/0 conn(0x7fd0bc0d2c30 legacy=0x7fd0bc0ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fd0c2800640 1 -- 172.21.15.49:0/236849071 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc0d3880 con 0x7fd0bc0d1300 2024-04-15T15:18:52.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fb829800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb82406f9b0 0x7fb82406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fb829800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb8240cd1e0 con 0x7fb824204690 2024-04-15T15:18:52.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fb829800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8240cfba0 con 0x7fb8080043a0 2024-04-15T15:18:52.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fb829800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb8240cd360 con 0x7fb82406f9b0 2024-04-15T15:18:52.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7fac5d600640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] conn(0x7fac38074bc0 0x7fac38076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.805+0000 7faff5400640 1 -- 172.21.15.49:0/2879622558 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8035020 con 0x7faff815ec40 2024-04-15T15:18:52.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7fac57e00640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fac3806c940 0x7fac3806ee00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fac5806a750 tx=0x7fac5806f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7faff5400640 1 -- 172.21.15.49:0/2879622558 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe8031640 con 0x7faff815ec40 2024-04-15T15:18:52.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7faff5400640 1 -- 172.21.15.49:0/2879622558 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8035020 con 0x7faff815ec40 2024-04-15T15:18:52.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7fb823400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb824204690 0x7fb824201b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7fb823400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb824204690 0x7fb824201b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58480/0 (socket says 172.21.15.49:58480) 2024-04-15T15:18:52.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7fb823400640 1 -- 172.21.15.49:0/109489198 learned_addr learned my addr 172.21.15.49:0/109489198 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7fb823e00640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8080043a0 0x7fb8240cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7ff187400640 1 Processor -- start 2024-04-15T15:18:52.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7f2d1e000640 1 -- 172.21.15.49:0/3699760388 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d180052d0 con 0x7f2d280d0450 2024-04-15T15:18:52.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7f2d1e000640 1 -- 172.21.15.49:0/3699760388 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d18005470 con 0x7f2d280d0450 2024-04-15T15:18:52.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7f2d1e000640 1 -- 172.21.15.49:0/3699760388 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18007240 con 0x7f2d280d0450 2024-04-15T15:18:52.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7ff187400640 1 -- start start 2024-04-15T15:18:52.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7fc6e6a00640 1 -- 172.21.15.49:0/2649509414 <== osd.2 v2:172.21.15.49:6814/1321297224 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6e000af30 con 0x7fc6d0074b60 2024-04-15T15:18:52.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7fb828e00640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb82406f9b0 0x7fb82406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d281457a0 con 0x7f2d280d0450 2024-04-15T15:18:52.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.806+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d28145d60 con 0x7f2d280d0450 2024-04-15T15:18:52.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7faffd400640 1 -- 172.21.15.49:0/2879622558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faff815ec40 msgr2=0x7faff80d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7fa99b400640 1 --2- 172.21.15.49:0/1988089397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c0d4360 0x7fa99c0d2c30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa994013db0 tx=0x7fa994034190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4acd41c8a9e4de server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7faffd400640 1 --2- 172.21.15.49:0/2879622558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faff815ec40 0x7faff80d4360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009870 tx=0x7fafe802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7fac5d600640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] conn(0x7fac38074bc0 0x7fac38076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d2806f970 con 0x7f2d280d0450 2024-04-15T15:18:52.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7fb823400640 1 -- 172.21.15.49:0/109489198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb82406f9b0 msgr2=0x7fb82406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7fb823400640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb82406f9b0 0x7fb82406fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7fb823400640 1 -- 172.21.15.49:0/109489198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8080043a0 msgr2=0x7fb8240cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7fb823400640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8080043a0 0x7fb8240cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7fb823400640 1 -- 172.21.15.49:0/109489198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb824202050 con 0x7fb824204690 2024-04-15T15:18:52.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7fb823e00640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8080043a0 0x7fb8240cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:52.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7fb823400640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb824204690 0x7fb824201b10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb814009be0 tx=0x7fb814004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7fd0c2800640 1 --2- 172.21.15.49:0/236849071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc0d1300 0x7fd0bc0d17b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009870 tx=0x7fd0b002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4ea765726587718 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7fd0c0a00640 1 -- 172.21.15.49:0/236849071 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0034070 con 0x7fd0bc0d1300 2024-04-15T15:18:52.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7fd0c0a00640 1 -- 172.21.15.49:0/236849071 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b0030840 con 0x7fd0bc0d1300 2024-04-15T15:18:52.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7fd0c0a00640 1 -- 172.21.15.49:0/236849071 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0030c10 con 0x7fd0bc0d1300 2024-04-15T15:18:52.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7fa99aa00640 1 -- 172.21.15.49:0/1988089397 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994041070 con 0x7fa99c0d4360 2024-04-15T15:18:52.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7fa99aa00640 1 -- 172.21.15.49:0/1988089397 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa99403c440 con 0x7fa99c0d4360 2024-04-15T15:18:52.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7fa99aa00640 1 -- 172.21.15.49:0/1988089397 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99403c810 con 0x7fa99c0d4360 2024-04-15T15:18:52.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7faffd400640 1 -- 172.21.15.49:0/2879622558 shutdown_connections 2024-04-15T15:18:52.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7faffd400640 1 --2- 172.21.15.49:0/2879622558 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faff80d2c30 0x7faff80ba390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7faffd400640 1 --2- 172.21.15.49:0/2879622558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faff815ec40 0x7faff80d4360 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.807+0000 7faffd400640 1 -- 172.21.15.49:0/2879622558 >> 172.21.15.49:0/2879622558 conn(0x7faff8073690 msgr2=0x7faff806cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7ff187400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff180162e90 0x7ff1800b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7ff187400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff180072810 0x7ff1800ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7ff187400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1800b01d0 con 0x7ff1800afd70 2024-04-15T15:18:52.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7ff187400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7ff1800b3900 con 0x7ff180072810 2024-04-15T15:18:52.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.808+0000 7ff187400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff1800b3be0 con 0x7ff180162e90 2024-04-15T15:18:52.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fb81be00640 1 -- 172.21.15.49:0/109489198 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814039050 con 0x7fb824204690 2024-04-15T15:18:52.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb824204c50 con 0x7fb824204690 2024-04-15T15:18:52.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb824205210 con 0x7fb824204690 2024-04-15T15:18:52.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fac54c00640 1 -- 172.21.15.49:0/1810286613 <== osd.0 v2:172.21.15.49:6808/2011863493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac4c017e50 con 0x7fac38074bc0 2024-04-15T15:18:52.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fb81be00640 1 -- 172.21.15.49:0/109489198 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb814045070 con 0x7fb824204690 2024-04-15T15:18:52.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fb81be00640 1 -- 172.21.15.49:0/109489198 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814009690 con 0x7fb824204690 2024-04-15T15:18:52.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fd0c4800640 1 -- 172.21.15.49:0/236849071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc0d1300 msgr2=0x7fd0bc0d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fd0c4800640 1 --2- 172.21.15.49:0/236849071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc0d1300 0x7fd0bc0d17b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009870 tx=0x7fd0b002b980 comp rx=0 tx=0).stop 2024-04-15T15:18:52.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fa9a3200640 1 -- 172.21.15.49:0/1988089397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c0d4360 msgr2=0x7fa99c0d2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fa9a3200640 1 --2- 172.21.15.49:0/1988089397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c0d4360 0x7fa99c0d2c30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa994013db0 tx=0x7fa994034190 comp rx=0 tx=0).stop 2024-04-15T15:18:52.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fa9a3200640 1 -- 172.21.15.49:0/1988089397 shutdown_connections 2024-04-15T15:18:52.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fa9a3200640 1 --2- 172.21.15.49:0/1988089397 >> v2:172.21.15.107:3300/0 conn(0x7fa99c0d1300 0x7fa99c0d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fa9a3200640 1 --2- 172.21.15.49:0/1988089397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c0d4360 0x7fa99c0d2c30 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa994013db0 tx=0x7fa994034190 comp rx=0 tx=0).stop 2024-04-15T15:18:52.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7fa9a3200640 1 -- 172.21.15.49:0/1988089397 >> 172.21.15.49:0/1988089397 conn(0x7fa99c073690 msgr2=0x7fa99c06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7faffd400640 1 -- 172.21.15.49:0/2879622558 shutdown_connections 2024-04-15T15:18:52.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.809+0000 7faffd400640 1 -- 172.21.15.49:0/2879622558 wait complete. 2024-04-15T15:18:52.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.810+0000 7ff17fe00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff1800afd70 0x7ff180072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44066/0 (socket says 172.21.15.49:44066) 2024-04-15T15:18:52.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.810+0000 7ff17fe00640 1 -- 172.21.15.49:0/1981241644 learned_addr learned my addr 172.21.15.49:0/1981241644 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:52.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.810+0000 7ff184a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff180162e90 0x7ff1800b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.810+0000 7ff185400640 1 --2- 172.21.15.49:0/1981241644 >> v2:172.21.15.107:3300/0 conn(0x7ff180072810 0x7ff1800ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.810+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7fc001030 con 0x7fb824204690 2024-04-15T15:18:52.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.810+0000 7fd0c4800640 1 -- 172.21.15.49:0/236849071 shutdown_connections 2024-04-15T15:18:52.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.810+0000 7fd0c4800640 1 --2- 172.21.15.49:0/236849071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc0d1300 0x7fd0bc0d17b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.810+0000 7fd0c4800640 1 --2- 172.21.15.49:0/236849071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc15ec40 0x7fd0bc0d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.810+0000 7fd0c4800640 1 -- 172.21.15.49:0/236849071 >> 172.21.15.49:0/236849071 conn(0x7fd0bc073690 msgr2=0x7fd0bc06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.810+0000 7fa9a3200640 1 -- 172.21.15.49:0/1988089397 shutdown_connections 2024-04-15T15:18:52.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.811+0000 7ff185400640 1 -- 172.21.15.49:0/1981241644 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff180162e90 msgr2=0x7ff1800b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:52.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.811+0000 7ff185400640 1 --2- 172.21.15.49:0/1981241644 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff180162e90 0x7ff1800b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.811+0000 7ff185400640 1 -- 172.21.15.49:0/1981241644 >> v1:172.21.15.49:6789/0 conn(0x7ff1800afd70 legacy=0x7ff180072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.811+0000 7ff185400640 1 -- 172.21.15.49:0/1981241644 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1800b3ec0 con 0x7ff180072810 2024-04-15T15:18:52.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.812+0000 7fa9a3200640 1 -- 172.21.15.49:0/1988089397 wait complete. 2024-04-15T15:18:52.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.812+0000 7fd0c4800640 1 -- 172.21.15.49:0/236849071 shutdown_connections 2024-04-15T15:18:52.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.812+0000 7fd0c4800640 1 -- 172.21.15.49:0/236849071 wait complete. 2024-04-15T15:18:52.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.812+0000 7faffd400640 1 Processor -- start 2024-04-15T15:18:52.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.812+0000 7ff185400640 1 --2- 172.21.15.49:0/1981241644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff180072810 0x7ff1800ce790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff178009d90 tx=0x7ff17802be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849c277f3d5bbb47 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.813+0000 7fb81be00640 1 -- 172.21.15.49:0/109489198 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb814005b90 con 0x7fb824204690 2024-04-15T15:18:52.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.813+0000 7ff17f400640 1 -- 172.21.15.49:0/1981241644 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178039070 con 0x7ff180072810 2024-04-15T15:18:52.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.813+0000 7ff17f400640 1 -- 172.21.15.49:0/1981241644 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17802cc00 con 0x7ff180072810 2024-04-15T15:18:52.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.813+0000 7ff17f400640 1 -- 172.21.15.49:0/1981241644 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178034440 con 0x7ff180072810 2024-04-15T15:18:52.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.813+0000 7f2d1e000640 1 -- 172.21.15.49:0/3699760388 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d180073e0 con 0x7f2d280d0450 2024-04-15T15:18:52.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.813+0000 7faffd400640 1 -- start start 2024-04-15T15:18:52.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7faffd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faff80d1300 0x7faff8160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7ff187400640 1 -- 172.21.15.49:0/1981241644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff180072810 msgr2=0x7ff1800ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7ff187400640 1 --2- 172.21.15.49:0/1981241644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff180072810 0x7ff1800ce790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff178009d90 tx=0x7ff17802be50 comp rx=0 tx=0).stop 2024-04-15T15:18:52.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7f2d1e000640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c9c0 0x7f2d0806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.814+0000 7fb81be00640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb80006c920 0x7fb80006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7fb81be00640 1 -- 172.21.15.49:0/109489198 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb8140ae3e0 con 0x7fb824204690 2024-04-15T15:18:52.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7fb81be00640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] conn(0x7fb800074b40 0x7fb800076f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7ff187400640 1 -- 172.21.15.49:0/1981241644 shutdown_connections 2024-04-15T15:18:52.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7fb81be00640 1 -- 172.21.15.49:0/109489198 --> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb800077610 con 0x7fb800074b40 2024-04-15T15:18:52.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7fb81be00640 1 -- 172.21.15.49:0/109489198 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb8140ae7d0 con 0x7fb824204690 2024-04-15T15:18:52.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7fd0c4800640 1 Processor -- start 2024-04-15T15:18:52.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7ff187400640 1 --2- 172.21.15.49:0/1981241644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff180072810 0x7ff1800ce790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7ff187400640 1 --2- 172.21.15.49:0/1981241644 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff180162e90 0x7ff1800b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7ff187400640 1 -- 172.21.15.49:0/1981241644 >> 172.21.15.49:0/1981241644 conn(0x7ff1800b2460 msgr2=0x7ff1800b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7f2d1e000640 1 -- 172.21.15.49:0/3699760388 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2d1808b9c0 con 0x7f2d280d0450 2024-04-15T15:18:52.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7f2d1e000640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] conn(0x7f2d08074be0 0x7f2d08077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7fd0c4800640 1 -- start start 2024-04-15T15:18:52.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7ff187400640 1 -- 172.21.15.49:0/1981241644 shutdown_connections 2024-04-15T15:18:52.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7fb823e00640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb80006c920 0x7fb80006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.816+0000 7fb828e00640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] conn(0x7fb800074b40 0x7fb800076f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.815+0000 7faffd400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faff80d2c30 0x7faff8154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.816+0000 7f2d1e000640 1 -- 172.21.15.49:0/3699760388 --> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d18018e50 con 0x7f2d08074be0 2024-04-15T15:18:52.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.816+0000 7f2d1e000640 1 -- 172.21.15.49:0/3699760388 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2d1808bdb0 con 0x7f2d280d0450 2024-04-15T15:18:52.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.816+0000 7f2d25e00640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c9c0 0x7f2d0806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.816+0000 7faffd400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faff815ec40 0x7faff80b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.816+0000 7fb823e00640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb80006c920 0x7fb80006ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb80c007fd0 tx=0x7fb80c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7fb828e00640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] conn(0x7fb800074b40 0x7fb800076f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7faffd400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faff80d3e40 con 0x7faff80d2c30 2024-04-15T15:18:52.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7f2d25400640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] conn(0x7f2d08074be0 0x7f2d08077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7faff6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faff80d1300 0x7faff8160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7faff5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faff80d2c30 0x7faff8154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7faff5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faff80d2c30 0x7faff8154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58488/0 (socket says 172.21.15.49:58488) 2024-04-15T15:18:52.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7faff5e00640 1 -- 172.21.15.49:0/3158647360 learned_addr learned my addr 172.21.15.49:0/3158647360 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faff80d3b60 con 0x7faff80d1300 2024-04-15T15:18:52.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7faff80d1790 con 0x7faff815ec40 2024-04-15T15:18:52.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.816+0000 7ff187400640 1 -- 172.21.15.49:0/1981241644 wait complete. 2024-04-15T15:18:52.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7fd0c4800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d1300 0x7fd0bc155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7f2d25e00640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c9c0 0x7f2d0806ee80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2d20000ba0 tx=0x7f2d20011040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7fd0c4800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc0d2c30 0x7fd0bc160a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7fa9a3200640 1 Processor -- start 2024-04-15T15:18:52.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.817+0000 7faff7200640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faff815ec40 0x7faff80b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7fd0c4800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0bc15ec40 0x7fd0bc15f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7fd0c1e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d1300 0x7fd0bc155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7fa9a3200640 1 -- start start 2024-04-15T15:18:52.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7fd0c1e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d1300 0x7fd0bc155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46056/0 (socket says 172.21.15.49:46056) 2024-04-15T15:18:52.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7fd0c4800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc0b78f0 con 0x7fd0bc15ec40 2024-04-15T15:18:52.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7fd0c4800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc0d3e40 con 0x7fd0bc0d2c30 2024-04-15T15:18:52.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7fd0c4800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd0bc0b7a70 con 0x7fd0bc0d1300 2024-04-15T15:18:52.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.819+0000 7fd0c1e00640 1 -- 172.21.15.49:0/2299777541 learned_addr learned my addr 172.21.15.49:0/2299777541 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.819+0000 7ff187400640 1 Processor -- start 2024-04-15T15:18:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.819+0000 7fb81be00640 1 -- 172.21.15.49:0/109489198 <== osd.3 v2:172.21.15.49:6822/1234122534 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb800077610 con 0x7fb800074b40 2024-04-15T15:18:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7faff5e00640 1 -- 172.21.15.49:0/3158647360 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faff815ec40 msgr2=0x7faff80b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7faff5e00640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faff815ec40 0x7faff80b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7faff5e00640 1 -- 172.21.15.49:0/3158647360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faff80d1300 msgr2=0x7faff8160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7faff5e00640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faff80d1300 0x7faff8160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.818+0000 7faff5e00640 1 -- 172.21.15.49:0/3158647360 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff8154590 con 0x7faff80d2c30 2024-04-15T15:18:52.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.819+0000 7ff187400640 1 -- start start 2024-04-15T15:18:52.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.819+0000 7fd0c1400640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc0d2c30 0x7fd0bc160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.819+0000 7f2d25400640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] conn(0x7f2d08074be0 0x7f2d08077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.819+0000 7fd0c2800640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0bc15ec40 0x7fd0bc15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7fa9a3200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa99c0d1300 0x7fa99c201150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7fa9a3200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c15ec40 0x7fa99c0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7fa9a3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa99c0ce4d0 0x7fa99c0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7fa9a3200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c0b7a70 con 0x7fa99c0d1300 2024-04-15T15:18:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.819+0000 7faff5e00640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faff80d2c30 0x7faff8154050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fafe400d830 tx=0x7fafe400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7fafeea00640 1 -- 172.21.15.49:0/3158647360 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe400b660 con 0x7faff80d2c30 2024-04-15T15:18:52.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7ff187400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff180072810 0x7ff1801ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7ff187400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1800afd70 0x7ff18015c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7ff187400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff180162e90 0x7ff1801ee480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7ff187400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800cee50 con 0x7ff180162e90 2024-04-15T15:18:52.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7ff187400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800cecd0 con 0x7ff180072810 2024-04-15T15:18:52.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7ff187400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff1800cefd0 con 0x7ff1800afd70 2024-04-15T15:18:52.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7fa9a3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c0b78f0 con 0x7fa99c0ce4d0 2024-04-15T15:18:52.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff8161f10 con 0x7faff80d2c30 2024-04-15T15:18:52.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff8161150 con 0x7faff80d2c30 2024-04-15T15:18:52.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7fa99b400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c15ec40 0x7fa99c0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7f2d1e000640 1 -- 172.21.15.49:0/3699760388 <== osd.1 v2:172.21.15.49:6800/4210904815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d18018e50 con 0x7f2d08074be0 2024-04-15T15:18:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7fa9a3200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c0b7bf0 con 0x7fa99c15ec40 2024-04-15T15:18:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7fd0c2800640 1 -- 172.21.15.49:0/2299777541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d1300 msgr2=0x7fd0bc155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7fd0c2800640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d1300 0x7fd0bc155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7fd0c2800640 1 -- 172.21.15.49:0/2299777541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc0d2c30 msgr2=0x7fd0bc160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7fd0c2800640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc0d2c30 0x7fd0bc160a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.820+0000 7fd0c2800640 1 -- 172.21.15.49:0/2299777541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc1558d0 con 0x7fd0bc15ec40 2024-04-15T15:18:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7fa99b400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c15ec40 0x7fa99c0cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46078/0 (socket says 172.21.15.49:46078) 2024-04-15T15:18:52.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7fa99b400640 1 -- 172.21.15.49:0/131116297 learned_addr learned my addr 172.21.15.49:0/131116297 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7ff17fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1800afd70 0x7ff18015c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7ff17fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1800afd70 0x7ff18015c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46072/0 (socket says 172.21.15.49:46072) 2024-04-15T15:18:52.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.821+0000 7ff17fe00640 1 -- 172.21.15.49:0/3696941947 learned_addr learned my addr 172.21.15.49:0/3696941947 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:52.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.822+0000 7fafeea00640 1 -- 172.21.15.49:0/3158647360 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe4019070 con 0x7faff80d2c30 2024-04-15T15:18:52.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.822+0000 7fa99b400640 1 -- 172.21.15.49:0/131116297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa99c0ce4d0 msgr2=0x7fa99c0c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.822+0000 7fa99b400640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa99c0ce4d0 0x7fa99c0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.822+0000 7fa99b400640 1 -- 172.21.15.49:0/131116297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa99c0d1300 msgr2=0x7fa99c201150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.822+0000 7fa99b400640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa99c0d1300 0x7fa99c201150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.822+0000 7fa99b400640 1 -- 172.21.15.49:0/131116297 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c0ced40 con 0x7fa99c15ec40 2024-04-15T15:18:52.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.822+0000 7fafeea00640 1 -- 172.21.15.49:0/3158647360 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe400ba20 con 0x7faff80d2c30 2024-04-15T15:18:52.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.822+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faff80cb760 con 0x7faff80d2c30 2024-04-15T15:18:52.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.823+0000 7fa99b400640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c15ec40 0x7fa99c0cdf90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9940090b0 tx=0x7fa9940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.823+0000 7ff17fe00640 1 -- 172.21.15.49:0/3696941947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff180072810 msgr2=0x7ff1801ebfc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-15T15:18:52.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.823+0000 7ff17fe00640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff180072810 0x7ff1801ebfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.823+0000 7ff17fe00640 1 -- 172.21.15.49:0/3696941947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff180162e90 msgr2=0x7ff1801ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.823+0000 7ff17fe00640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff180162e90 0x7ff1801ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.823+0000 7ff17fe00640 1 -- 172.21.15.49:0/3696941947 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1801ec6c0 con 0x7ff1800afd70 2024-04-15T15:18:52.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.824+0000 7fa993e00640 1 -- 172.21.15.49:0/131116297 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994041070 con 0x7fa99c15ec40 2024-04-15T15:18:52.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.824+0000 7ff17fe00640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1800afd70 0x7ff18015c8e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00c4d0 tx=0x7ff16c00c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.824+0000 7fa993e00640 1 -- 172.21.15.49:0/131116297 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa99403a4c0 con 0x7fa99c15ec40 2024-04-15T15:18:52.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.824+0000 7fa993e00640 1 -- 172.21.15.49:0/131116297 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99403a8b0 con 0x7fa99c15ec40 2024-04-15T15:18:52.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.824+0000 7fd0c2800640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0bc15ec40 0x7fd0bc15f020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0033000 tx=0x7fd0b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.825+0000 7fd0b6000640 1 -- 172.21.15.49:0/2299777541 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0048070 con 0x7fd0bc15ec40 2024-04-15T15:18:52.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.825+0000 7fd0b6000640 1 -- 172.21.15.49:0/2299777541 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b0030e20 con 0x7fd0bc15ec40 2024-04-15T15:18:52.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.825+0000 7fd0b6000640 1 -- 172.21.15.49:0/2299777541 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0034040 con 0x7fd0bc15ec40 2024-04-15T15:18:52.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.825+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0bc15f990 con 0x7fd0bc15ec40 2024-04-15T15:18:52.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.825+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0bc160230 con 0x7fd0bc15ec40 2024-04-15T15:18:52.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.827+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd0bc15fd90 con 0x7fd0bc15ec40 2024-04-15T15:18:52.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.824+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c0cea60 con 0x7fa99c15ec40 2024-04-15T15:18:52.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.827+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c15f750 con 0x7fa99c15ec40 2024-04-15T15:18:52.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.825+0000 7ff17cc00640 1 -- 172.21.15.49:0/3696941947 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c017070 con 0x7ff1800afd70 2024-04-15T15:18:52.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.825+0000 7ff17cc00640 1 -- 172.21.15.49:0/3696941947 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16c005ce0 con 0x7ff1800afd70 2024-04-15T15:18:52.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.825+0000 7ff17cc00640 1 -- 172.21.15.49:0/3696941947 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c007270 con 0x7ff1800afd70 2024-04-15T15:18:52.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.827+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1801ec9a0 con 0x7ff1800afd70 2024-04-15T15:18:52.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.828+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa99c068c80 con 0x7fa99c15ec40 2024-04-15T15:18:52.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.828+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1800a9e70 con 0x7ff1800afd70 2024-04-15T15:18:52.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.825+0000 7fafeea00640 1 -- 172.21.15.49:0/3158647360 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafe400bbc0 con 0x7faff80d2c30 2024-04-15T15:18:52.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.826+0000 7fafeea00640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fafd806c8f0 0x7fafd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.826+0000 7fafeea00640 1 -- 172.21.15.49:0/3158647360 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fafe408bf30 con 0x7faff80d2c30 2024-04-15T15:18:52.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.827+0000 7fafeea00640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] conn(0x7fafd8074b70 0x7fafd8076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.827+0000 7fafeea00640 1 -- 172.21.15.49:0/3158647360 --> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fafd8077640 con 0x7fafd8074b70 2024-04-15T15:18:52.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.827+0000 7fafeea00640 1 -- 172.21.15.49:0/3158647360 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fafe4093050 con 0x7faff80d2c30 2024-04-15T15:18:52.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.827+0000 7faff6800640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fafd806c8f0 0x7fafd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.828+0000 7faff6800640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fafd806c8f0 0x7fafd806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fafe8036000 tx=0x7fafe803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.829+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff180068c80 con 0x7ff1800afd70 2024-04-15T15:18:52.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.829+0000 7faff7200640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] conn(0x7fafd8074b70 0x7fafd8076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.830+0000 7faff7200640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] conn(0x7fafd8074b70 0x7fafd8076f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.829+0000 7fd0b6000640 1 -- 172.21.15.49:0/2299777541 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0b004b070 con 0x7fd0bc15ec40 2024-04-15T15:18:52.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.831+0000 7fd0b6000640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd0a006c940 0x7fd0a006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.830+0000 7fa993e00640 1 -- 172.21.15.49:0/131116297 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa994044070 con 0x7fa99c15ec40 2024-04-15T15:18:52.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.831+0000 7fa993e00640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa97806c8f0 0x7fa97806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.831+0000 7fa993e00640 1 -- 172.21.15.49:0/131116297 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa9940ae760 con 0x7fa99c15ec40 2024-04-15T15:18:52.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.831+0000 7fa993e00640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] conn(0x7fa978074b70 0x7fa978076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.832+0000 7fd0b6000640 1 -- 172.21.15.49:0/2299777541 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd0b0079710 con 0x7fd0bc15ec40 2024-04-15T15:18:52.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.831+0000 7fa993e00640 1 -- 172.21.15.49:0/131116297 --> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa99403c440 con 0x7fa978074b70 2024-04-15T15:18:52.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.832+0000 7fa993e00640 1 -- 172.21.15.49:0/131116297 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa9940aeb50 con 0x7fa99c15ec40 2024-04-15T15:18:52.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.832+0000 7fa99be00640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa97806c8f0 0x7fa97806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.832+0000 7ff17cc00640 1 -- 172.21.15.49:0/3696941947 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff16c01a070 con 0x7ff1800afd70 2024-04-15T15:18:52.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.833+0000 7fa99be00640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa97806c8f0 0x7fa97806edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa984006fd0 tx=0x7fa984006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.833+0000 7fa9a1200640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] conn(0x7fa978074b70 0x7fa978076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.833+0000 7fd0b6000640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] conn(0x7fd0a0074ae0 0x7fd0a0076f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.833+0000 7fd0b6000640 1 -- 172.21.15.49:0/2299777541 --> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0b0030840 con 0x7fd0a0074ae0 2024-04-15T15:18:52.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.833+0000 7fd0b6000640 1 -- 172.21.15.49:0/2299777541 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd0b0077910 con 0x7fd0bc15ec40 2024-04-15T15:18:52.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.833+0000 7fd0c1e00640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd0a006c940 0x7fd0a006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.833+0000 7fd0c1400640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] conn(0x7fd0a0074ae0 0x7fd0a0076f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.834+0000 7fafeea00640 1 -- 172.21.15.49:0/3158647360 <== osd.6 v2:172.21.15.107:6809/4081027681 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafd8077640 con 0x7fafd8074b70 2024-04-15T15:18:52.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.833+0000 7ff17cc00640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff16006c8f0 0x7ff16006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.834+0000 7ff184a00640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff16006c8f0 0x7ff16006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.834+0000 7fd0c1e00640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd0a006c940 0x7fd0a006ee00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80094c0 tx=0x7fd0b800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.834+0000 7fa9a1200640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] conn(0x7fa978074b70 0x7fa978076f90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.834+0000 7ff17cc00640 1 -- 172.21.15.49:0/3696941947 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff16c08af80 con 0x7ff1800afd70 2024-04-15T15:18:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.835+0000 7fd0c1400640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] conn(0x7fd0a0074ae0 0x7fd0a0076f00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.835+0000 7ff17cc00640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] conn(0x7ff160074b10 0x7ff160076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.835+0000 7ff17cc00640 1 -- 172.21.15.49:0/3696941947 --> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1600775e0 con 0x7ff160074b10 2024-04-15T15:18:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.835+0000 7ff17cc00640 1 -- 172.21.15.49:0/3696941947 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff16c08b370 con 0x7ff1800afd70 2024-04-15T15:18:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.835+0000 7ff184a00640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff16006c8f0 0x7ff16006edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff180165ca0 tx=0x7ff174008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.836+0000 7ff185400640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] conn(0x7ff160074b10 0x7ff160076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.837+0000 7ff185400640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] conn(0x7ff160074b10 0x7ff160076f30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.837+0000 7fa993e00640 1 -- 172.21.15.49:0/131116297 <== osd.5 v2:172.21.15.107:6808/1708671582 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa99403c440 con 0x7fa978074b70 2024-04-15T15:18:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.839+0000 7fd0b6000640 1 -- 172.21.15.49:0/2299777541 <== osd.4 v2:172.21.15.107:6800/3185658252 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0b0030840 con 0x7fd0a0074ae0 2024-04-15T15:18:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.839+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 --> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6f00c9d70 con 0x7fc6d0074b60 2024-04-15T15:18:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.840+0000 7ff17cc00640 1 -- 172.21.15.49:0/3696941947 <== osd.7 v2:172.21.15.107:6820/876829831 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1600775e0 con 0x7ff160074b10 2024-04-15T15:18:52.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.841+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 --> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac500dfa30 con 0x7fac38074bc0 2024-04-15T15:18:52.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.842+0000 7fac54c00640 1 -- 172.21.15.49:0/1810286613 <== osd.0 v2:172.21.15.49:6808/2011863493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fac500dfa30 con 0x7fac38074bc0 2024-04-15T15:18:52.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.842+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] conn(0x7fac38074bc0 msgr2=0x7fac38076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.842+0000 7fac5e000640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] conn(0x7fac38074bc0 0x7fac38076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.844+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fac3806c940 msgr2=0x7fac3806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.844+0000 7fac5e000640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fac3806c940 0x7fac3806ee00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fac5806a750 tx=0x7fac5806f040 comp rx=0 tx=0).stop 2024-04-15T15:18:52.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.844+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac5000c0a0 msgr2=0x7fac5004e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.844+0000 7fac5e000640 1 --2- 172.21.15.49:0/1810286613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac5000c0a0 0x7fac5004e1d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00b500 tx=0x7fac4c00b9c0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.844+0000 7fac5d600640 1 -- 172.21.15.49:0/1810286613 reap_dead start 2024-04-15T15:18:52.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.844+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 shutdown_connections 2024-04-15T15:18:52.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.844+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 >> 172.21.15.49:0/1810286613 conn(0x7fac50009fb0 msgr2=0x7fac50034c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.844+0000 7fc6e6a00640 1 -- 172.21.15.49:0/2649509414 <== osd.2 v2:172.21.15.49:6814/1321297224 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc6f00c9d70 con 0x7fc6d0074b60 2024-04-15T15:18:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.847+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 shutdown_connections 2024-04-15T15:18:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.847+0000 7fac5e000640 1 -- 172.21.15.49:0/1810286613 wait complete. 2024-04-15T15:18:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.847+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] conn(0x7fc6d0074b60 msgr2=0x7fc6d0076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.847+0000 7fc6f5a00640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] conn(0x7fc6d0074b60 0x7fc6d0076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.848+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc6d006c940 msgr2=0x7fc6d006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.848+0000 7fc6f5a00640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc6d006c940 0x7fc6d006ee00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8005c00 tx=0x7fc6e8009040 comp rx=0 tx=0).stop 2024-04-15T15:18:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.848+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f01669d0 msgr2=0x7fc6f01fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.848+0000 7fc6f5a00640 1 --2- 172.21.15.49:0/2649509414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f01669d0 0x7fc6f01fbe60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009890 tx=0x7fc6e00046c0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.848+0000 7fc6ef400640 1 -- 172.21.15.49:0/2649509414 reap_dead start 2024-04-15T15:18:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.848+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 shutdown_connections 2024-04-15T15:18:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.848+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 >> 172.21.15.49:0/2649509414 conn(0x7fc6f00d1970 msgr2=0x7fc6f0165e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.848+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 shutdown_connections 2024-04-15T15:18:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.848+0000 7fc6f5a00640 1 -- 172.21.15.49:0/2649509414 wait complete. 2024-04-15T15:18:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.851+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 --> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d2814cb20 con 0x7f2d08074be0 2024-04-15T15:18:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.852+0000 7f2d1e000640 1 -- 172.21.15.49:0/3699760388 <== osd.1 v2:172.21.15.49:6800/4210904815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2d2814cb20 con 0x7f2d08074be0 2024-04-15T15:18:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.852+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] conn(0x7f2d08074be0 msgr2=0x7f2d08077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.852+0000 7f2d2ca00640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] conn(0x7f2d08074be0 0x7f2d08077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.853+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c9c0 msgr2=0x7f2d0806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.853+0000 7f2d2ca00640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c9c0 0x7f2d0806ee80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2d20000ba0 tx=0x7f2d20011040 comp rx=0 tx=0).stop 2024-04-15T15:18:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.853+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d280d0450 msgr2=0x7f2d28146320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.853+0000 7f2d2ca00640 1 --2- 172.21.15.49:0/3699760388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d280d0450 0x7f2d28146320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800c3c0 tx=0x7f2d1800c880 comp rx=0 tx=0).stop 2024-04-15T15:18:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.853+0000 7f2d26800640 1 -- 172.21.15.49:0/3699760388 reap_dead start 2024-04-15T15:18:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.854+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 shutdown_connections 2024-04-15T15:18:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.854+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 >> 172.21.15.49:0/3699760388 conn(0x7f2d2806b930 msgr2=0x7f2d280b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.855+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 --> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa99c0cb750 con 0x7fa978074b70 2024-04-15T15:18:52.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.856+0000 7fa993e00640 1 -- 172.21.15.49:0/131116297 <== osd.5 v2:172.21.15.107:6808/1708671582 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa99c0cb750 con 0x7fa978074b70 2024-04-15T15:18:52.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.857+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 >> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] conn(0x7fa978074b70 msgr2=0x7fa978076f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.857+0000 7fa9a3200640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] conn(0x7fa978074b70 0x7fa978076f90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.857+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa97806c8f0 msgr2=0x7fa97806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.857+0000 7fa9a3200640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa97806c8f0 0x7fa97806edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa984006fd0 tx=0x7fa984006bc0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.857+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c15ec40 msgr2=0x7fa99c0cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.857+0000 7fa9a3200640 1 --2- 172.21.15.49:0/131116297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa99c15ec40 0x7fa99c0cdf90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9940090b0 tx=0x7fa9940046c0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.857+0000 7fa9a1200640 1 -- 172.21.15.49:0/131116297 reap_dead start 2024-04-15T15:18:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.857+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 shutdown_connections 2024-04-15T15:18:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.857+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 >> 172.21.15.49:0/131116297 conn(0x7fa99c073690 msgr2=0x7fa99c0d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.858+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 shutdown_connections 2024-04-15T15:18:52.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.858+0000 7f2d2ca00640 1 -- 172.21.15.49:0/3699760388 wait complete. 2024-04-15T15:18:52.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.859+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 --> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7fc002dd0 con 0x7fb800074b40 2024-04-15T15:18:52.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.860+0000 7fb81be00640 1 -- 172.21.15.49:0/109489198 <== osd.3 v2:172.21.15.49:6822/1234122534 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb7fc002dd0 con 0x7fb800074b40 2024-04-15T15:18:52.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.861+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 shutdown_connections 2024-04-15T15:18:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.861+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] conn(0x7fb800074b40 msgr2=0x7fb800076f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.861+0000 7fb829800640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] conn(0x7fb800074b40 0x7fb800076f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.861+0000 7fa9a3200640 1 -- 172.21.15.49:0/131116297 wait complete. 2024-04-15T15:18:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.862+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb80006c920 msgr2=0x7fb80006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.862+0000 7fb829800640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb80006c920 0x7fb80006ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb80c007fd0 tx=0x7fb80c009040 comp rx=0 tx=0).stop 2024-04-15T15:18:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.862+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb824204690 msgr2=0x7fb824201b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.862+0000 7fb829800640 1 --2- 172.21.15.49:0/109489198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb824204690 0x7fb824201b10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb814009be0 tx=0x7fb814004730 comp rx=0 tx=0).stop 2024-04-15T15:18:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.862+0000 7fb828e00640 1 -- 172.21.15.49:0/109489198 reap_dead start 2024-04-15T15:18:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.862+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 shutdown_connections 2024-04-15T15:18:52.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.862+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 >> 172.21.15.49:0/109489198 conn(0x7fb824071e30 msgr2=0x7fb82414f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.863+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 shutdown_connections 2024-04-15T15:18:52.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.863+0000 7fb829800640 1 -- 172.21.15.49:0/109489198 wait complete. 2024-04-15T15:18:52.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.867+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 --> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faff815f270 con 0x7fafd8074b70 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.868+0000 7fafeea00640 1 -- 172.21.15.49:0/3158647360 <== osd.6 v2:172.21.15.107:6809/4081027681 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faff815f270 con 0x7fafd8074b70 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.869+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 >> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] conn(0x7fafd8074b70 msgr2=0x7fafd8076f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.869+0000 7faffd400640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] conn(0x7fafd8074b70 0x7fafd8076f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.869+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fafd806c8f0 msgr2=0x7fafd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.869+0000 7faffd400640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fafd806c8f0 0x7fafd806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fafe8036000 tx=0x7fafe803c040 comp rx=0 tx=0).stop 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.869+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faff80d2c30 msgr2=0x7faff8154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.869+0000 7faffd400640 1 --2- 172.21.15.49:0/3158647360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faff80d2c30 0x7faff8154050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fafe400d830 tx=0x7fafe400dcf0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.869+0000 7faff7200640 1 -- 172.21.15.49:0/3158647360 reap_dead start 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.871+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 shutdown_connections 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.871+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 >> 172.21.15.49:0/3158647360 conn(0x7faff8073690 msgr2=0x7faff815df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.874+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 shutdown_connections 2024-04-15T15:18:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.874+0000 7faffd400640 1 -- 172.21.15.49:0/3158647360 wait complete. 2024-04-15T15:18:52.891 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:52.891 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-15T15:18:52.892 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2024-04-15T15:18:52.893 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-15T15:18:52.893 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:52.894 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-15T15:18:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.881+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 --> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff18006ed60 con 0x7ff160074b10 2024-04-15T15:18:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.882+0000 7ff17cc00640 1 -- 172.21.15.49:0/3696941947 <== osd.7 v2:172.21.15.107:6820/876829831 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff18006ed60 con 0x7ff160074b10 2024-04-15T15:18:52.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.882+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 --> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd0bc068c80 con 0x7fd0a0074ae0 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.882+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 >> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] conn(0x7ff160074b10 msgr2=0x7ff160076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7ff187400640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] conn(0x7ff160074b10 0x7ff160076f30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff16006c8f0 msgr2=0x7ff16006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7ff187400640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff16006c8f0 0x7ff16006edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff180165ca0 tx=0x7ff174008040 comp rx=0 tx=0).stop 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1800afd70 msgr2=0x7ff18015c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7ff187400640 1 --2- 172.21.15.49:0/3696941947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1800afd70 0x7ff18015c8e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00c4d0 tx=0x7ff16c00c990 comp rx=0 tx=0).stop 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7ff185400640 1 -- 172.21.15.49:0/3696941947 reap_dead start 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 shutdown_connections 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 >> 172.21.15.49:0/3696941947 conn(0x7ff1800b2460 msgr2=0x7ff1800b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 shutdown_connections 2024-04-15T15:18:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7ff187400640 1 -- 172.21.15.49:0/3696941947 wait complete. 2024-04-15T15:18:52.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.883+0000 7fd0b6000640 1 -- 172.21.15.49:0/2299777541 <== osd.4 v2:172.21.15.107:6800/3185658252 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd0bc068c80 con 0x7fd0a0074ae0 2024-04-15T15:18:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.884+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 >> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] conn(0x7fd0a0074ae0 msgr2=0x7fd0a0076f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.884+0000 7fd0c4800640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] conn(0x7fd0a0074ae0 0x7fd0a0076f00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.884+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd0a006c940 msgr2=0x7fd0a006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.884+0000 7fd0c4800640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd0a006c940 0x7fd0a006ee00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80094c0 tx=0x7fd0b800b040 comp rx=0 tx=0).stop 2024-04-15T15:18:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.884+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0bc15ec40 msgr2=0x7fd0bc15f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.884+0000 7fd0c4800640 1 --2- 172.21.15.49:0/2299777541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0bc15ec40 0x7fd0bc15f020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0033000 tx=0x7fd0b0004730 comp rx=0 tx=0).stop 2024-04-15T15:18:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.884+0000 7fd0c2800640 1 -- 172.21.15.49:0/2299777541 reap_dead start 2024-04-15T15:18:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.885+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 shutdown_connections 2024-04-15T15:18:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.885+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 >> 172.21.15.49:0/2299777541 conn(0x7fd0bc073690 msgr2=0x7fd0bc0c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:52.899 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:52.899 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-15T15:18:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.885+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 shutdown_connections 2024-04-15T15:18:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:52.885+0000 7fd0c4800640 1 -- 172.21.15.49:0/2299777541 wait complete. 2024-04-15T15:18:52.901 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:52.901 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-15T15:18:52.902 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:52.903 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-15T15:18:52.906 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:52.907 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-15T15:18:52.919 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:52.919 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-15T15:18:53.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.209+0000 7f7651000640 1 Processor -- start 2024-04-15T15:18:53.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.209+0000 7fd6b9000640 1 Processor -- start 2024-04-15T15:18:53.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.210+0000 7fd6b9000640 1 -- start start 2024-04-15T15:18:53.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.211+0000 7f0f91200640 1 Processor -- start 2024-04-15T15:18:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.210+0000 7f7651000640 1 -- start start 2024-04-15T15:18:53.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.211+0000 7f0f91200640 1 -- start start 2024-04-15T15:18:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7f7651000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f764c15a910 0x7f764c0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7f7651000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f764c0ceaa0 0x7f764c06ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7f7651000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c0cef40 con 0x7f764c06f8a0 2024-04-15T15:18:53.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7f7651000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f764c0cf220 con 0x7f764c15a910 2024-04-15T15:18:53.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7f7651000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f764c0cf500 con 0x7f764c0ceaa0 2024-04-15T15:18:53.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.213+0000 7f764a800640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f764c06f8a0 0x7f764c0b3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:46298/0 (socket says 172.21.15.49:46298) 2024-04-15T15:18:53.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.211+0000 7f199ce00640 1 Processor -- start 2024-04-15T15:18:53.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7f199ce00640 1 -- start start 2024-04-15T15:18:53.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7f199ce00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 0x7f19981633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7f199ce00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f19981669d0 0x7f19980cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7f199ce00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19980d1170 con 0x7f19980cf450 2024-04-15T15:18:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7f199ce00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f1998167900 con 0x7f19981669d0 2024-04-15T15:18:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7f199ce00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f19980cf8b0 con 0x7f1998163f90 2024-04-15T15:18:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7fd6b9000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b415a910 0x7fd6b40cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7fd6b9000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd6b4070640 0x7fd6b40b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7fd6b9000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b40cef40 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7fd6b9000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b40cf220 con 0x7fd6b4070640 2024-04-15T15:18:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.212+0000 7fd6b9000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b40cf500 con 0x7fd6b415a910 2024-04-15T15:18:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.213+0000 7fd6b1e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b415a910 0x7fd6b40cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.213+0000 7fd6b1e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b415a910 0x7fd6b40cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46080/0 (socket says 172.21.15.49:46080) 2024-04-15T15:18:53.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.213+0000 7fd6b1e00640 1 -- 172.21.15.49:0/3891549469 learned_addr learned my addr 172.21.15.49:0/3891549469 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.213+0000 7fd6b2800640 1 --2- 172.21.15.49:0/3891549469 >> v2:172.21.15.49:3300/0 conn(0x7fd6b4070640 0x7fd6b40b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.213+0000 7f764a800640 1 -- 172.21.15.49:0/2964520267 learned_addr learned my addr 172.21.15.49:0/2964520267 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:53.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.213+0000 7f7649e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f764c15a910 0x7f764c0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.213+0000 7f7649400640 1 --2- 172.21.15.49:0/2964520267 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f764c0ceaa0 0x7f764c06ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7f0f91200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0f8c070640 0x7f0f8c0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7fd6b2800640 1 -- 172.21.15.49:0/3891549469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b415a910 msgr2=0x7fd6b40cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:53.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7fd6b2800640 1 --2- 172.21.15.49:0/3891549469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b415a910 0x7fd6b40cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7fd6b2800640 1 -- 172.21.15.49:0/3891549469 >> v1:172.21.15.107:6789/0 conn(0x7fd6b40ceaa0 legacy=0x7fd6b406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7fd6b2800640 1 -- 172.21.15.49:0/3891549469 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b40cf7e0 con 0x7fd6b4070640 2024-04-15T15:18:53.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.215+0000 7f0f89e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0f8c15a910 0x7f0f8c06ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44078/0 (socket says 172.21.15.49:44078) 2024-04-15T15:18:53.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.215+0000 7f1996800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f19980cf450 0x7f199806cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44068/0 (socket says 172.21.15.49:44068) 2024-04-15T15:18:53.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.215+0000 7f1996800640 1 -- 172.21.15.49:0/2334347678 learned_addr learned my addr 172.21.15.49:0/2334347678 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:53.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.215+0000 7f0f89e00640 1 -- 172.21.15.49:0/2717675670 learned_addr learned my addr 172.21.15.49:0/2717675670 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:53.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.215+0000 7f0f91200640 1 -- 172.21.15.49:0/2717675670 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f8c0cef40 con 0x7f0f8c15a910 2024-04-15T15:18:53.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.215+0000 7f1995400640 1 --2- 172.21.15.49:0/2334347678 >> v2:172.21.15.107:3300/0 conn(0x7f19981669d0 0x7f19980cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.215+0000 7f1995e00640 1 --2- 172.21.15.49:0/2334347678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 0x7f19981633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7f7649e00640 1 -- 172.21.15.49:0/2964520267 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f764c0ceaa0 msgr2=0x7f764c06ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:53.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.215+0000 7f0f8a800640 1 --2- 172.21.15.49:0/2717675670 >> v2:172.21.15.49:3300/0 conn(0x7f0f8c070640 0x7f0f8c0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7f7649e00640 1 --2- 172.21.15.49:0/2964520267 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f764c0ceaa0 0x7f764c06ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7f7649e00640 1 -- 172.21.15.49:0/2964520267 >> v1:172.21.15.107:6789/0 conn(0x7f764c06f8a0 legacy=0x7f764c0b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.214+0000 7f7649e00640 1 -- 172.21.15.49:0/2964520267 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c0cf7e0 con 0x7f764c15a910 2024-04-15T15:18:53.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.216+0000 7f54aee00640 1 Processor -- start 2024-04-15T15:18:53.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.216+0000 7f0f91200640 1 -- 172.21.15.49:0/2717675670 --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f8c0cf220 con 0x7f0f8c0ceaa0 2024-04-15T15:18:53.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.216+0000 7f0f91200640 1 -- 172.21.15.49:0/2717675670 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f0f8c0cf500 con 0x7f0f8c070640 2024-04-15T15:18:53.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.216+0000 7f54aee00640 1 -- start start 2024-04-15T15:18:53.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.216+0000 7f7649e00640 1 --2- 172.21.15.49:0/2964520267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f764c15a910 0x7f764c0cffe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f763c00a8c0 tx=0x7f763c02c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8efda9c323611a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.217+0000 7f0f88a00640 1 -- 172.21.15.49:0/2717675670 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2958076601 0 0) 0x7f0f8c0cf220 con 0x7f0f8c0ceaa0 2024-04-15T15:18:53.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.217+0000 7f0f8a800640 1 -- 172.21.15.49:0/2717675670 >> v1:172.21.15.107:6789/0 conn(0x7f0f8c0ceaa0 legacy=0x7f0f8c0b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.217+0000 7f0f8a800640 1 -- 172.21.15.49:0/2717675670 >> v1:172.21.15.49:6789/0 conn(0x7f0f8c15a910 legacy=0x7f0f8c06ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.217+0000 7f0f8a800640 1 -- 172.21.15.49:0/2717675670 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8c0cf7e0 con 0x7f0f8c070640 2024-04-15T15:18:53.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.215+0000 7f1995400640 1 -- 172.21.15.49:0/2334347678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 msgr2=0x7f19981633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:53.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.215+0000 7f1995400640 1 --2- 172.21.15.49:0/2334347678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 0x7f19981633e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.216+0000 7f1995400640 1 -- 172.21.15.49:0/2334347678 >> v1:172.21.15.49:6789/0 conn(0x7f19980cf450 legacy=0x7f199806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.216+0000 7f1995400640 1 -- 172.21.15.49:0/2334347678 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1998167060 con 0x7f19981669d0 2024-04-15T15:18:53.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.217+0000 7f7648a00640 1 -- 172.21.15.49:0/2964520267 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c038070 con 0x7f764c15a910 2024-04-15T15:18:53.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.217+0000 7f7648a00640 1 -- 172.21.15.49:0/2964520267 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f763c031710 con 0x7f764c15a910 2024-04-15T15:18:53.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.217+0000 7f7648a00640 1 -- 172.21.15.49:0/2964520267 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c031ae0 con 0x7f764c15a910 2024-04-15T15:18:53.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.217+0000 7f1995400640 1 --2- 172.21.15.49:0/2334347678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19981669d0 0x7f19980cef10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1988009d90 tx=0x7f198802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef2773f151bfdfc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.218+0000 7f1994a00640 1 -- 172.21.15.49:0/2334347678 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988039070 con 0x7f19981669d0 2024-04-15T15:18:53.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.218+0000 7f54aee00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a815ec60 0x7f54a80ba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f7651000640 1 -- 172.21.15.49:0/2964520267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f764c15a910 msgr2=0x7f764c0cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.218+0000 7f0f8a800640 1 --2- 172.21.15.49:0/2717675670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c070640 0x7f0f8c0cffe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009d90 tx=0x7f0f8402bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6457ea59fe80415 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f7651000640 1 --2- 172.21.15.49:0/2964520267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f764c15a910 0x7f764c0cffe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f763c00a8c0 tx=0x7f763c02c790 comp rx=0 tx=0).stop 2024-04-15T15:18:53.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.218+0000 7f1994a00640 1 -- 172.21.15.49:0/2334347678 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198802cc50 con 0x7f19981669d0 2024-04-15T15:18:53.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.218+0000 7f1994a00640 1 -- 172.21.15.49:0/2334347678 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988034440 con 0x7f19981669d0 2024-04-15T15:18:53.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f97f6e00640 1 Processor -- start 2024-04-15T15:18:53.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.218+0000 7f54aee00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a80b8f30 con 0x7f54a80d3c90 2024-04-15T15:18:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f7651000640 1 -- 172.21.15.49:0/2964520267 shutdown_connections 2024-04-15T15:18:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f7651000640 1 --2- 172.21.15.49:0/2964520267 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f764c0ceaa0 0x7f764c06ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f7651000640 1 --2- 172.21.15.49:0/2964520267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f764c15a910 0x7f764c0cffe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f7651000640 1 -- 172.21.15.49:0/2964520267 >> 172.21.15.49:0/2964520267 conn(0x7f764c0b28b0 msgr2=0x7f764c0b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f7651000640 1 -- 172.21.15.49:0/2964520267 shutdown_connections 2024-04-15T15:18:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f0f88a00640 1 -- 172.21.15.49:0/2717675670 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84039070 con 0x7f0f8c070640 2024-04-15T15:18:53.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f199ce00640 1 -- 172.21.15.49:0/2334347678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19981669d0 msgr2=0x7f19980cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f199ce00640 1 --2- 172.21.15.49:0/2334347678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19981669d0 0x7f19980cef10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1988009d90 tx=0x7f198802bea0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f7651000640 1 -- 172.21.15.49:0/2964520267 wait complete. 2024-04-15T15:18:53.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.218+0000 7f54aee00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a80b9210 con 0x7f54a80b63d0 2024-04-15T15:18:53.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.218+0000 7f54aee00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f54a80b94f0 con 0x7f54a815ec60 2024-04-15T15:18:53.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.220+0000 7f0f88a00640 1 -- 172.21.15.49:0/2717675670 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8402ca30 con 0x7f0f8c070640 2024-04-15T15:18:53.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7f97f6e00640 1 -- start start 2024-04-15T15:18:53.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.219+0000 7fd6b2800640 1 --2- 172.21.15.49:0/3891549469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6b4070640 0x7fd6b40b3760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0090e0 tx=0x7fd69c02c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64a882dd53eb611 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.220+0000 7fd6b0a00640 1 -- 172.21.15.49:0/3891549469 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c036020 con 0x7fd6b4070640 2024-04-15T15:18:53.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.220+0000 7fd6b0a00640 1 -- 172.21.15.49:0/3891549469 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c032890 con 0x7fd6b4070640 2024-04-15T15:18:53.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.220+0000 7fd6b0a00640 1 -- 172.21.15.49:0/3891549469 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c032c60 con 0x7fd6b4070640 2024-04-15T15:18:53.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.220+0000 7f54ace00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f54a80d3c90 0x7f54a80b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44084/0 (socket says 172.21.15.49:44084) 2024-04-15T15:18:53.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.220+0000 7f54ace00640 1 -- 172.21.15.49:0/2106368159 learned_addr learned my addr 172.21.15.49:0/2106368159 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:53.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.220+0000 7f54a7e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a815ec60 0x7f54a80ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.220+0000 7f0f88a00640 1 -- 172.21.15.49:0/2717675670 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8402ce00 con 0x7f0f8c070640 2024-04-15T15:18:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7f0f91200640 1 -- 172.21.15.49:0/2717675670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c070640 msgr2=0x7f0f8c0cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7f0f91200640 1 --2- 172.21.15.49:0/2717675670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c070640 0x7f0f8c0cffe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009d90 tx=0x7f0f8402bd00 comp rx=0 tx=0).stop 2024-04-15T15:18:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7f0f91200640 1 -- 172.21.15.49:0/2717675670 shutdown_connections 2024-04-15T15:18:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7f0f91200640 1 --2- 172.21.15.49:0/2717675670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c070640 0x7f0f8c0cffe0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7f0f91200640 1 -- 172.21.15.49:0/2717675670 >> 172.21.15.49:0/2717675670 conn(0x7f0f8c0b28b0 msgr2=0x7f0f8c0b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f0f91200640 1 -- 172.21.15.49:0/2717675670 shutdown_connections 2024-04-15T15:18:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f0f91200640 1 -- 172.21.15.49:0/2717675670 wait complete. 2024-04-15T15:18:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f199ce00640 1 -- 172.21.15.49:0/2334347678 shutdown_connections 2024-04-15T15:18:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f199ce00640 1 --2- 172.21.15.49:0/2334347678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19981669d0 0x7f19980cef10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f199ce00640 1 --2- 172.21.15.49:0/2334347678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 0x7f19981633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f199ce00640 1 -- 172.21.15.49:0/2334347678 >> 172.21.15.49:0/2334347678 conn(0x7f19980d1970 msgr2=0x7f19980d48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7f54a7400640 1 -- 172.21.15.49:0/2106368159 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1926676470 0 0) 0x7f54a80b8f30 con 0x7f54a80d3c90 2024-04-15T15:18:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7f54a7e00640 1 -- 172.21.15.49:0/2106368159 >> v1:172.21.15.107:6789/0 conn(0x7f54a80b63d0 legacy=0x7f54a80b7a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-15T15:18:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7f54a7e00640 1 -- 172.21.15.49:0/2106368159 >> v1:172.21.15.49:6789/0 conn(0x7f54a80d3c90 legacy=0x7f54a80b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7f54a7e00640 1 -- 172.21.15.49:0/2106368159 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a80b97d0 con 0x7f54a815ec60 2024-04-15T15:18:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7fd6b9000640 1 -- 172.21.15.49:0/3891549469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6b4070640 msgr2=0x7fd6b40b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7fd6b9000640 1 --2- 172.21.15.49:0/3891549469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6b4070640 0x7fd6b40b3760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0090e0 tx=0x7fd69c02c930 comp rx=0 tx=0).stop 2024-04-15T15:18:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7fd6b9000640 1 -- 172.21.15.49:0/3891549469 shutdown_connections 2024-04-15T15:18:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7fd6b9000640 1 --2- 172.21.15.49:0/3891549469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6b4070640 0x7fd6b40b3760 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7fd6b9000640 1 --2- 172.21.15.49:0/3891549469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b415a910 0x7fd6b40cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.221+0000 7fd6b9000640 1 -- 172.21.15.49:0/3891549469 >> 172.21.15.49:0/3891549469 conn(0x7fd6b40b28b0 msgr2=0x7fd6b40b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7fd6b9000640 1 -- 172.21.15.49:0/3891549469 shutdown_connections 2024-04-15T15:18:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7fd6b9000640 1 -- 172.21.15.49:0/3891549469 wait complete. 2024-04-15T15:18:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f7651000640 1 Processor -- start 2024-04-15T15:18:53.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f7651000640 1 -- start start 2024-04-15T15:18:53.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f97f6e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 0x7f97f00cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f97f6e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f97f0072100 0x7f97f006ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f97f6e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f97f006f8a0 0x7f97f0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f97f6e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f97f00cf220 con 0x7f97f0072100 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f97f6e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f97f00cf500 con 0x7f97f006f8a0 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f97f6e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f97f00cf7e0 con 0x7f97f00ceaa0 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.223+0000 7f97efe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 0x7f97f00cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.223+0000 7f97efe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 0x7f97f00cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46126/0 (socket says 172.21.15.49:46126) 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.223+0000 7f97efe00640 1 -- 172.21.15.49:0/36533329 learned_addr learned my addr 172.21.15.49:0/36533329 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.223+0000 7f97ef400640 1 --2- 172.21.15.49:0/36533329 >> v2:172.21.15.49:3300/0 conn(0x7f97f0072100 0x7f97f006ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.223+0000 7f97f4e00640 1 --2- 172.21.15.49:0/36533329 >> v2:172.21.15.107:3300/0 conn(0x7f97f006f8a0 0x7f97f0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.223+0000 7f97efe00640 1 -- 172.21.15.49:0/36533329 >> v2:172.21.15.107:3300/0 conn(0x7f97f006f8a0 msgr2=0x7f97f0070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.223+0000 7f97efe00640 1 --2- 172.21.15.49:0/36533329 >> v2:172.21.15.107:3300/0 conn(0x7f97f006f8a0 0x7f97f0070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.223+0000 7f97efe00640 1 -- 172.21.15.49:0/36533329 >> v2:172.21.15.49:3300/0 conn(0x7f97f0072100 msgr2=0x7f97f006ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f199ce00640 1 -- 172.21.15.49:0/2334347678 shutdown_connections 2024-04-15T15:18:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.222+0000 7f199ce00640 1 -- 172.21.15.49:0/2334347678 wait complete. 2024-04-15T15:18:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.223+0000 7f97efe00640 1 --2- 172.21.15.49:0/36533329 >> v2:172.21.15.49:3300/0 conn(0x7f97f0072100 0x7f97f006ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.223+0000 7f97efe00640 1 -- 172.21.15.49:0/36533329 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f00cfac0 con 0x7f97f00ceaa0 2024-04-15T15:18:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.224+0000 7f97ef400640 1 --2- 172.21.15.49:0/36533329 >> v2:172.21.15.49:3300/0 conn(0x7f97f0072100 0x7f97f006ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.224+0000 7fd6b9000640 1 Processor -- start 2024-04-15T15:18:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.224+0000 7f54a7e00640 1 --2- 172.21.15.49:0/2106368159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a815ec60 0x7f54a80ba270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54900097d0 tx=0x7f549002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e74f5dd879fbb02 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7f199ce00640 1 Processor -- start 2024-04-15T15:18:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7f7651000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f764c06f8a0 0x7f764c1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7f7651000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764c0ceaa0 0x7f764c1fe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7f199ce00640 1 -- start start 2024-04-15T15:18:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7f0f91200640 1 Processor -- start 2024-04-15T15:18:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7fd6b9000640 1 -- start start 2024-04-15T15:18:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7f7651000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f764c15a910 0x7f764c1529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7f7651000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f764c15acf0 con 0x7f764c0ceaa0 2024-04-15T15:18:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7f0f91200640 1 -- start start 2024-04-15T15:18:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7f7651000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f764c072100 con 0x7f764c15a910 2024-04-15T15:18:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.225+0000 7f7651000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f764c150ad0 con 0x7f764c06f8a0 2024-04-15T15:18:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f7649400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764c0ceaa0 0x7f764c1fe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f7649400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764c0ceaa0 0x7f764c1fe6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58536/0 (socket says 172.21.15.49:58536) 2024-04-15T15:18:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f7649400640 1 -- 172.21.15.49:0/404977601 learned_addr learned my addr 172.21.15.49:0/404977601 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f764a800640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f764c15a910 0x7f764c1529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7fd6b9000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b4070640 0x7fd6b41ed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7fd6b9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6b40ceaa0 0x7fd6b4153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f7649400640 1 -- 172.21.15.49:0/404977601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f764c06f8a0 msgr2=0x7f764c1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f7649400640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f764c06f8a0 0x7f764c1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f54a7400640 1 -- 172.21.15.49:0/2106368159 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490035020 con 0x7f54a815ec60 2024-04-15T15:18:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f7649400640 1 -- 172.21.15.49:0/404977601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f764c15a910 msgr2=0x7f764c1529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f7649400640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f764c15a910 0x7f764c1529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f7649400640 1 -- 172.21.15.49:0/404977601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c150db0 con 0x7f764c0ceaa0 2024-04-15T15:18:53.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7f54a7400640 1 -- 172.21.15.49:0/2106368159 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5490031810 con 0x7f54a815ec60 2024-04-15T15:18:53.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7f54a7400640 1 -- 172.21.15.49:0/2106368159 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490031be0 con 0x7f54a815ec60 2024-04-15T15:18:53.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.226+0000 7fd6b9000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6b415a910 0x7fd6b4154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7f7649400640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764c0ceaa0 0x7f764c1fe6f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f763400eb70 tx=0x7f763400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7fd6b9000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b415acf0 con 0x7fd6b415a910 2024-04-15T15:18:53.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7fd6b2800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6b415a910 0x7fd6b4154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7fd6b1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6b40ceaa0 0x7fd6b4153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7fd6b9000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4072100 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7f97efe00640 1 --2- 172.21.15.49:0/36533329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 0x7f97f00cffe0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f97f00b5800 tx=0x7f97d802b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa90adbc76e376 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7fd6b9000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b4150130 con 0x7fd6b4070640 2024-04-15T15:18:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7fd6b2800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6b415a910 0x7fd6b4154030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58544/0 (socket says 172.21.15.49:58544) 2024-04-15T15:18:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7fd6b1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6b40ceaa0 0x7fd6b4153af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:50468/0 (socket says 172.21.15.49:50468) 2024-04-15T15:18:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7fd6b2800640 1 -- 172.21.15.49:0/2731535458 learned_addr learned my addr 172.21.15.49:0/2731535458 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.227+0000 7fd6b1e00640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b4070640 0x7fd6b41ed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f0f91200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f8c070640 0x7f0f8c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f7642000640 1 -- 172.21.15.49:0/404977601 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634017070 con 0x7f764c0ceaa0 2024-04-15T15:18:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f7642000640 1 -- 172.21.15.49:0/404977601 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f763400ed80 con 0x7f764c0ceaa0 2024-04-15T15:18:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f7642000640 1 -- 172.21.15.49:0/404977601 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763400ce50 con 0x7f764c0ceaa0 2024-04-15T15:18:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f54aee00640 1 -- 172.21.15.49:0/2106368159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a815ec60 msgr2=0x7f54a80ba270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f54aee00640 1 --2- 172.21.15.49:0/2106368159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a815ec60 0x7f54a80ba270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54900097d0 tx=0x7f549002b8b0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f54aee00640 1 -- 172.21.15.49:0/2106368159 shutdown_connections 2024-04-15T15:18:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f54aee00640 1 --2- 172.21.15.49:0/2106368159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a815ec60 0x7f54a80ba270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f54aee00640 1 -- 172.21.15.49:0/2106368159 >> 172.21.15.49:0/2106368159 conn(0x7f54a8073480 msgr2=0x7f54a8073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f0f91200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f8c0ceaa0 0x7f0f8c155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f0f91200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15a910 0x7f0f8c1fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f199ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19980cf450 0x7f19980ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f199ce00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 0x7f19980cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f199ce00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19981669d0 0x7f1998164d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f199ce00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f19980d26c0 con 0x7f19981669d0 2024-04-15T15:18:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f199ce00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1998164370 con 0x7f19980cf450 2024-04-15T15:18:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f199ce00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1998167620 con 0x7f1998163f90 2024-04-15T15:18:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f9a5e400640 1 Processor -- start 2024-04-15T15:18:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f54aee00640 1 -- 172.21.15.49:0/2106368159 shutdown_connections 2024-04-15T15:18:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f97eea00640 1 -- 172.21.15.49:0/36533329 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8035020 con 0x7f97f00ceaa0 2024-04-15T15:18:53.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f1996800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19981669d0 0x7f1998164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f1996800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19981669d0 0x7f1998164d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58548/0 (socket says 172.21.15.49:58548) 2024-04-15T15:18:53.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f1996800640 1 -- 172.21.15.49:0/3805218984 learned_addr learned my addr 172.21.15.49:0/3805218984 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f0f91200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c15acf0 con 0x7f0f8c15a910 2024-04-15T15:18:53.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f0f91200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c06f8a0 con 0x7f0f8c0ceaa0 2024-04-15T15:18:53.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f0f91200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8c1558d0 con 0x7f0f8c070640 2024-04-15T15:18:53.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f0f89e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f8c070640 0x7f0f8c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f54aee00640 1 -- 172.21.15.49:0/2106368159 wait complete. 2024-04-15T15:18:53.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7f7651000640 1 -- 172.21.15.49:0/404977601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764c151090 con 0x7f764c0ceaa0 2024-04-15T15:18:53.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f0f8a800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15a910 0x7f0f8c1fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f0f8a800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15a910 0x7f0f8c1fb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58552/0 (socket says 172.21.15.49:58552) 2024-04-15T15:18:53.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f0f8a800640 1 -- 172.21.15.49:0/3914071520 learned_addr learned my addr 172.21.15.49:0/3914071520 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f0f89400640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f8c0ceaa0 0x7f0f8c155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f1995400640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 0x7f19980cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f97eea00640 1 -- 172.21.15.49:0/36533329 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d8031870 con 0x7f97f00ceaa0 2024-04-15T15:18:53.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f97eea00640 1 -- 172.21.15.49:0/36533329 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8031c40 con 0x7f97f00ceaa0 2024-04-15T15:18:53.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7fd6b1400640 1 -- 172.21.15.49:0/2731535458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b4070640 msgr2=0x7fd6b41ed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7fd6b1400640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b4070640 0x7fd6b41ed5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7fd6b1400640 1 -- 172.21.15.49:0/2731535458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6b415a910 msgr2=0x7fd6b4154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7fd6b1400640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6b415a910 0x7fd6b4154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.228+0000 7fd6b1400640 1 -- 172.21.15.49:0/2731535458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b415da00 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f9a5e400640 1 -- start start 2024-04-15T15:18:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.229+0000 7f1995e00640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19980cf450 0x7f19980ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f1995400640 1 -- 172.21.15.49:0/3805218984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19980cf450 msgr2=0x7f19980ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f1995400640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19980cf450 0x7f19980ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f1995400640 1 -- 172.21.15.49:0/3805218984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19981669d0 msgr2=0x7f1998164d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f1995400640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19981669d0 0x7f1998164d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f1995400640 1 -- 172.21.15.49:0/3805218984 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19981505e0 con 0x7f1998163f90 2024-04-15T15:18:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7fd6b1400640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6b40ceaa0 0x7fd6b4153af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00c770 tx=0x7fd6ac00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.231+0000 7f7651000640 1 -- 172.21.15.49:0/404977601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764c151650 con 0x7f764c0ceaa0 2024-04-15T15:18:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f0f8a800640 1 -- 172.21.15.49:0/3914071520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f8c070640 msgr2=0x7f0f8c159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f0f8a800640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f8c070640 0x7f0f8c159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f0f8a800640 1 -- 172.21.15.49:0/3914071520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f8c0ceaa0 msgr2=0x7f0f8c155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f0f8a800640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f8c0ceaa0 0x7f0f8c155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f0f8a800640 1 -- 172.21.15.49:0/3914071520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8c155b00 con 0x7f0f8c15a910 2024-04-15T15:18:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.231+0000 7f0f8a800640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15a910 0x7f0f8c1fb6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009eb0 tx=0x7f0f84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7fd6aa000640 1 -- 172.21.15.49:0/2731535458 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac003b70 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7fd6b9000640 1 -- 172.21.15.49:0/2731535458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b4150360 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.231+0000 7fd6b9000640 1 -- 172.21.15.49:0/2731535458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b4150920 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f1995400640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 0x7f19980cae50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1988033000 tx=0x7f1988004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f54aee00640 1 Processor -- start 2024-04-15T15:18:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f97f6e00640 1 -- 172.21.15.49:0/36533329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 msgr2=0x7f97f00cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.230+0000 7f97f6e00640 1 --2- 172.21.15.49:0/36533329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 0x7f97f00cffe0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f97f00b5800 tx=0x7f97d802b910 comp rx=0 tx=0).stop 2024-04-15T15:18:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.231+0000 7fd6aa000640 1 -- 172.21.15.49:0/2731535458 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ac003d10 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.231+0000 7fd6aa000640 1 -- 172.21.15.49:0/2731535458 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac00aa30 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f0f82000640 1 -- 172.21.15.49:0/3914071520 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84044030 con 0x7f0f8c15a910 2024-04-15T15:18:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f198e000640 1 -- 172.21.15.49:0/3805218984 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988044020 con 0x7f1998163f90 2024-04-15T15:18:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f198e000640 1 -- 172.21.15.49:0/3805218984 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198802ca50 con 0x7f1998163f90 2024-04-15T15:18:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f198e000640 1 -- 172.21.15.49:0/3805218984 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988039040 con 0x7f1998163f90 2024-04-15T15:18:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f199ce00640 1 -- 172.21.15.49:0/3805218984 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19980cbe70 con 0x7f1998163f90 2024-04-15T15:18:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.231+0000 7f9a5e400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9a580d1550 0x7f9a580d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.231+0000 7f9a5e400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a580b78f0 con 0x7f9a580d32e0 2024-04-15T15:18:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.231+0000 7f9a5e400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a58162d40 con 0x7f9a5815eab0 2024-04-15T15:18:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.231+0000 7f9a5e400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f9a580d1ef0 con 0x7f9a580d1550 2024-04-15T15:18:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f54aee00640 1 -- start start 2024-04-15T15:18:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.233+0000 7fd6b9000640 1 -- 172.21.15.49:0/2731535458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68c005290 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f0f82000640 1 -- 172.21.15.49:0/3914071520 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8403a040 con 0x7f0f8c15a910 2024-04-15T15:18:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f0f82000640 1 -- 172.21.15.49:0/3914071520 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84044030 con 0x7f0f8c15a910 2024-04-15T15:18:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f0f91200640 1 -- 172.21.15.49:0/3914071520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8c155de0 con 0x7f0f8c15a910 2024-04-15T15:18:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.233+0000 7f0f91200640 1 -- 172.21.15.49:0/3914071520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8c1fbc00 con 0x7f0f8c15a910 2024-04-15T15:18:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.234+0000 7f97f6e00640 1 -- 172.21.15.49:0/36533329 shutdown_connections 2024-04-15T15:18:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.234+0000 7f97f6e00640 1 --2- 172.21.15.49:0/36533329 >> v2:172.21.15.107:3300/0 conn(0x7f97f006f8a0 0x7f97f0070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.234+0000 7f97f6e00640 1 --2- 172.21.15.49:0/36533329 >> v2:172.21.15.49:3300/0 conn(0x7f97f0072100 0x7f97f006ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.234+0000 7f97f6e00640 1 --2- 172.21.15.49:0/36533329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 0x7f97f00cffe0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.234+0000 7f97f6e00640 1 -- 172.21.15.49:0/36533329 >> 172.21.15.49:0/36533329 conn(0x7f97f00b28b0 msgr2=0x7f97f00b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.234+0000 7f9a5da00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9a580d32e0 0x7f9a580d3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44088/0 (socket says 172.21.15.49:44088) 2024-04-15T15:18:53.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.234+0000 7f9a5da00640 1 -- 172.21.15.49:0/3592782489 learned_addr learned my addr 172.21.15.49:0/3592782489 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:53.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.233+0000 7f199ce00640 1 -- 172.21.15.49:0/3805218984 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19980cc430 con 0x7f1998163f90 2024-04-15T15:18:53.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.232+0000 7f7651000640 1 -- 172.21.15.49:0/404977601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764c06e4e0 con 0x7f764c0ceaa0 2024-04-15T15:18:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.235+0000 7f97f6e00640 1 -- 172.21.15.49:0/36533329 shutdown_connections 2024-04-15T15:18:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.235+0000 7f199ce00640 1 -- 172.21.15.49:0/3805218984 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19980cb680 con 0x7f1998163f90 2024-04-15T15:18:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.234+0000 7f9a57e00640 1 --2- 172.21.15.49:0/3592782489 >> v2:172.21.15.49:3300/0 conn(0x7f9a580d1550 0x7f9a580d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.235+0000 7f54aee00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a80b63d0 0x7f54a81fd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f7642000640 1 -- 172.21.15.49:0/404977601 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f763401a070 con 0x7f764c0ceaa0 2024-04-15T15:18:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f97f6e00640 1 -- 172.21.15.49:0/36533329 wait complete. 2024-04-15T15:18:53.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f9a57e00640 1 -- 172.21.15.49:0/3592782489 >> v1:172.21.15.107:6789/0 conn(0x7f9a5815eab0 legacy=0x7f9a58162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f9a57e00640 1 -- 172.21.15.49:0/3592782489 >> v1:172.21.15.49:6789/0 conn(0x7f9a580d32e0 legacy=0x7f9a580d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f9a57e00640 1 -- 172.21.15.49:0/3592782489 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a58163010 con 0x7f9a580d1550 2024-04-15T15:18:53.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.237+0000 7fd6aa000640 1 -- 172.21.15.49:0/2731535458 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6ac00abd0 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f0f91200640 1 -- 172.21.15.49:0/3914071520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f64005290 con 0x7f0f8c15a910 2024-04-15T15:18:53.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.238+0000 7f0f82000640 1 -- 172.21.15.49:0/3914071520 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f84047070 con 0x7f0f8c15a910 2024-04-15T15:18:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f54aee00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54a80d3c90 0x7f54a80caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f54aee00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54a815ec60 0x7f54a80c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f54aee00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f54a80ba7b0 con 0x7f54a815ec60 2024-04-15T15:18:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f54aee00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54a815f040 con 0x7f54a80d3c90 2024-04-15T15:18:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f54aee00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f54a80ba930 con 0x7f54a80b63d0 2024-04-15T15:18:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f549fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54a80d3c90 0x7f54a80caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f549fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54a80d3c90 0x7f54a80caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:50488/0 (socket says 172.21.15.49:50488) 2024-04-15T15:18:53.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.236+0000 7f549fe00640 1 -- 172.21.15.49:0/2476728807 learned_addr learned my addr 172.21.15.49:0/2476728807 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.237+0000 7f54ace00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54a815ec60 0x7f54a80c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.238+0000 7f54a7e00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a80b63d0 0x7f54a81fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.238+0000 7f198e000640 1 -- 172.21.15.49:0/3805218984 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1988047020 con 0x7f1998163f90 2024-04-15T15:18:53.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.238+0000 7f7642000640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f762806cae0 0x7f762806efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.239+0000 7f7649e00640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f762806cae0 0x7f762806efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.239+0000 7f54a7e00640 1 -- 172.21.15.49:0/2476728807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54a80d3c90 msgr2=0x7f54a80caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.239+0000 7f54a7e00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54a80d3c90 0x7f54a80caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.239+0000 7f54a7e00640 1 -- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54a815ec60 msgr2=0x7f54a80c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.239+0000 7f54a7e00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54a815ec60 0x7f54a80c7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.240+0000 7f54a7e00640 1 -- 172.21.15.49:0/2476728807 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a81fc580 con 0x7f54a80b63d0 2024-04-15T15:18:53.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.239+0000 7f0f82000640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0f6c06caf0 0x7f0f6c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.239+0000 7f198e000640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f197406c9f0 0x7f197406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.240+0000 7f198e000640 1 -- 172.21.15.49:0/3805218984 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f19880ae770 con 0x7f1998163f90 2024-04-15T15:18:53.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.240+0000 7f1995e00640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f197406c9f0 0x7f197406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.239+0000 7fd6aa000640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd69006c920 0x7fd69006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.239+0000 7fd6aa000640 1 -- 172.21.15.49:0/2731535458 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd6ac08bc70 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.240+0000 7f97f6e00640 1 Processor -- start 2024-04-15T15:18:53.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.240+0000 7f0f82000640 1 -- 172.21.15.49:0/3914071520 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0f840add30 con 0x7f0f8c15a910 2024-04-15T15:18:53.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.240+0000 7f7649e00640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f762806cae0 0x7f762806efa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f763c035000 tx=0x7f763c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.240+0000 7fd6b1e00640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd69006c920 0x7fd69006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.241+0000 7f9a57e00640 1 --2- 172.21.15.49:0/3592782489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a580d1550 0x7f9a580d19b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a440097a0 tx=0x7f9a4402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd64a04f5d2268a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.241+0000 7f0f89e00640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0f6c06caf0 0x7f0f6c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.241+0000 7f7642000640 1 -- 172.21.15.49:0/404977601 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7634090060 con 0x7f764c0ceaa0 2024-04-15T15:18:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.240+0000 7f54ace00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54a815ec60 0x7f54a80c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:18:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.241+0000 7f54a7e00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a80b63d0 0x7f54a81fd020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5490033000 tx=0x7f54900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.241+0000 7f97f6e00640 1 -- start start 2024-04-15T15:18:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.242+0000 7f1995e00640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f197406c9f0 0x7f197406eeb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1980006fd0 tx=0x7f1980008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.242+0000 7fd6b1e00640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd69006c920 0x7fd69006ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009ac0 tx=0x7fd6a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.242+0000 7f0f89e00640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0f6c06caf0 0x7f0f6c06efb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c000fb0 tx=0x7f0f7c007c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.242+0000 7f97f6e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f006f8a0 0x7f97f0150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.242+0000 7f97f6e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f0072100 0x7f97f0153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97f6e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 0x7f97f01514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97f6e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97f015a6a0 con 0x7f97f0072100 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97f6e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97f006e6f0 con 0x7f97f006f8a0 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97f6e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f97f015a820 con 0x7f97f00ceaa0 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97ef400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f0072100 0x7f97f0153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97ef400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f0072100 0x7f97f0153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58580/0 (socket says 172.21.15.49:58580) 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97ef400640 1 -- 172.21.15.49:0/636235461 learned_addr learned my addr 172.21.15.49:0/636235461 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97efe00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f006f8a0 0x7f97f0150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97f4e00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 0x7f97f01514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97ef400640 1 -- 172.21.15.49:0/636235461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 msgr2=0x7f97f01514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97ef400640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 0x7f97f01514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97ef400640 1 -- 172.21.15.49:0/636235461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f006f8a0 msgr2=0x7f97f0150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97ef400640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f006f8a0 0x7f97f0150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97ef400640 1 -- 172.21.15.49:0/636235461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f014f870 con 0x7f97f0072100 2024-04-15T15:18:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.243+0000 7f97efe00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f006f8a0 0x7f97f0150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.245+0000 7f97ef400640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f0072100 0x7f97f0153dc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f97e000dad0 tx=0x7f97e000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.245+0000 7f7642000640 1 -- 172.21.15.49:0/404977601 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f764c06e4e0 con 0x7f764c0ceaa0 2024-04-15T15:18:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.250+0000 7f97e7e00640 1 -- 172.21.15.49:0/636235461 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0008d80 con 0x7f97f0072100 2024-04-15T15:18:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.250+0000 7f97e7e00640 1 -- 172.21.15.49:0/636235461 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e00173f0 con 0x7f97f0072100 2024-04-15T15:18:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.250+0000 7f97e7e00640 1 -- 172.21.15.49:0/636235461 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e00177c0 con 0x7f97f0072100 2024-04-15T15:18:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.250+0000 7f9a57400640 1 -- 172.21.15.49:0/3592782489 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44034070 con 0x7f9a580d1550 2024-04-15T15:18:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.250+0000 7f9a57400640 1 -- 172.21.15.49:0/3592782489 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a44030610 con 0x7f9a580d1550 2024-04-15T15:18:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.250+0000 7f9a57400640 1 -- 172.21.15.49:0/3592782489 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440309e0 con 0x7f9a580d1550 2024-04-15T15:18:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.250+0000 7f97f6e00640 1 -- 172.21.15.49:0/636235461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f014fb50 con 0x7f97f0072100 2024-04-15T15:18:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.250+0000 7f97f6e00640 1 -- 172.21.15.49:0/636235461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f0150110 con 0x7f97f0072100 2024-04-15T15:18:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.251+0000 7f0f82000640 1 -- 172.21.15.49:0/3914071520 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f8407c1e0 con 0x7f0f8c15a910 2024-04-15T15:18:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.251+0000 7f9a5e400640 1 -- 172.21.15.49:0/3592782489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a580d1550 msgr2=0x7f9a580d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.251+0000 7f9a5e400640 1 --2- 172.21.15.49:0/3592782489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a580d1550 0x7f9a580d19b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a440097a0 tx=0x7f9a4402b960 comp rx=0 tx=0).stop 2024-04-15T15:18:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.251+0000 7f9a5e400640 1 -- 172.21.15.49:0/3592782489 shutdown_connections 2024-04-15T15:18:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.251+0000 7f9a5e400640 1 --2- 172.21.15.49:0/3592782489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a580d1550 0x7f9a580d19b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.251+0000 7f9a5e400640 1 -- 172.21.15.49:0/3592782489 >> 172.21.15.49:0/3592782489 conn(0x7f9a58073240 msgr2=0x7f9a58073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.252+0000 7f97f6e00640 1 -- 172.21.15.49:0/636235461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f0068c80 con 0x7f97f0072100 2024-04-15T15:18:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.252+0000 7f9a5e400640 1 -- 172.21.15.49:0/3592782489 shutdown_connections 2024-04-15T15:18:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.252+0000 7f9a5e400640 1 -- 172.21.15.49:0/3592782489 wait complete. 2024-04-15T15:18:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.253+0000 7f54a4c00640 1 -- 172.21.15.49:0/2476728807 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54900453f0 con 0x7f54a80b63d0 2024-04-15T15:18:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.253+0000 7f54a4c00640 1 -- 172.21.15.49:0/2476728807 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5490045590 con 0x7f54a80b63d0 2024-04-15T15:18:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.253+0000 7f54a4c00640 1 -- 172.21.15.49:0/2476728807 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490045980 con 0x7f54a80b63d0 2024-04-15T15:18:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.253+0000 7f97e7e00640 1 -- 172.21.15.49:0/636235461 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97e0017960 con 0x7f97f0072100 2024-04-15T15:18:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.254+0000 7f9a5e400640 1 Processor -- start 2024-04-15T15:18:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.255+0000 7f97e7e00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97cc06c9c0 0x7f97cc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.254+0000 7f9a5e400640 1 -- start start 2024-04-15T15:18:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.253+0000 7f198e000640 1 -- 172.21.15.49:0/3805218984 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19980cb680 con 0x7f1998163f90 2024-04-15T15:18:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.255+0000 7fd6aa000640 1 -- 172.21.15.49:0/2731535458 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6b415da00 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.256+0000 7f97e7e00640 1 -- 172.21.15.49:0/636235461 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f97e008bd90 con 0x7f97f0072100 2024-04-15T15:18:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.256+0000 7f9a5e400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a580d1550 0x7f9a581529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.256+0000 7f9a5e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a580d32e0 0x7f9a5814f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.256+0000 7f9a5e400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5815eab0 0x7f9a580b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.257+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a580d32e0 0x7f9a5814f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.257+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a580d32e0 0x7f9a5814f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:50502/0 (socket says 172.21.15.49:50502) 2024-04-15T15:18:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.257+0000 7f9a5d000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a580d1550 0x7f9a581529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.257+0000 7f9a5d000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a580d1550 0x7f9a581529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46192/0 (socket says 172.21.15.49:46192) 2024-04-15T15:18:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.257+0000 7f9a5e400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9a580d2ae0 con 0x7f9a5815eab0 2024-04-15T15:18:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.257+0000 7f9a57e00640 1 -- 172.21.15.49:0/3926901334 learned_addr learned my addr 172.21.15.49:0/3926901334 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.257+0000 7f9a5da00640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5815eab0 0x7f9a580b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.258+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9a580d2e50 con 0x7f9a580d32e0 2024-04-15T15:18:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.258+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5815ee90 con 0x7f9a580d1550 2024-04-15T15:18:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.259+0000 7f9a5da00640 1 -- 172.21.15.49:0/3926901334 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a580d1550 msgr2=0x7f9a581529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.259+0000 7f9a5da00640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a580d1550 0x7f9a581529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.259+0000 7f9a5da00640 1 -- 172.21.15.49:0/3926901334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a580d32e0 msgr2=0x7f9a5814f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.259+0000 7f9a5da00640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a580d32e0 0x7f9a5814f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.258+0000 7f54aee00640 1 -- 172.21.15.49:0/2476728807 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a80cc710 con 0x7f54a80b63d0 2024-04-15T15:18:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.258+0000 7f54aee00640 1 -- 172.21.15.49:0/2476728807 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a80cc9f0 con 0x7f54a80b63d0 2024-04-15T15:18:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.259+0000 7f54aee00640 1 -- 172.21.15.49:0/2476728807 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a8068c80 con 0x7f54a80b63d0 2024-04-15T15:18:53.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.260+0000 7f97efe00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97cc06c9c0 0x7f97cc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.261+0000 7f97efe00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97cc06c9c0 0x7f97cc06ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97d8036000 tx=0x7f97d803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.260+0000 7f9a5da00640 1 -- 172.21.15.49:0/3926901334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a580cd240 con 0x7f9a5815eab0 2024-04-15T15:18:53.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.262+0000 7f9a57e00640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a580d32e0 0x7f9a5814f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:53.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.263+0000 7f9a5da00640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5815eab0 0x7f9a580b8c50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000caa0 tx=0x7f9a5000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.263+0000 7f54a4c00640 1 -- 172.21.15.49:0/2476728807 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f549004c060 con 0x7f54a80b63d0 2024-04-15T15:18:53.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.264+0000 7f9a54c00640 1 -- 172.21.15.49:0/3926901334 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50017070 con 0x7f9a5815eab0 2024-04-15T15:18:53.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.264+0000 7f9a54c00640 1 -- 172.21.15.49:0/3926901334 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a5000aba0 con 0x7f9a5815eab0 2024-04-15T15:18:53.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.264+0000 7f9a54c00640 1 -- 172.21.15.49:0/3926901334 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50012400 con 0x7f9a5815eab0 2024-04-15T15:18:53.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.265+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a580cd500 con 0x7f9a5815eab0 2024-04-15T15:18:53.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.265+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a580cdac0 con 0x7f9a5815eab0 2024-04-15T15:18:53.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.266+0000 7f97e7e00640 1 -- 172.21.15.49:0/636235461 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97e0012070 con 0x7f97f0072100 2024-04-15T15:18:53.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.267+0000 7f54a4c00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f548406c9c0 0x7f548406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.267+0000 7f54a4c00640 1 -- 172.21.15.49:0/2476728807 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f54900ae110 con 0x7f54a80b63d0 2024-04-15T15:18:53.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.268+0000 7f549fe00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f548406c9c0 0x7f548406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.267+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a580755c0 con 0x7f9a5815eab0 2024-04-15T15:18:53.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.269+0000 7f549fe00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f548406c9c0 0x7f548406ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f54a8073030 tx=0x7f5498009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.269+0000 7f9a54c00640 1 -- 172.21.15.49:0/3926901334 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a5001a070 con 0x7f9a5815eab0 2024-04-15T15:18:53.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.272+0000 7f9a54c00640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9a3806c8f0 0x7f9a3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.272+0000 7f9a54c00640 1 -- 172.21.15.49:0/3926901334 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9a5008b8a0 con 0x7f9a5815eab0 2024-04-15T15:18:53.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.272+0000 7f9a5d000640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9a3806c8f0 0x7f9a3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.273+0000 7f9a5d000640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9a3806c8f0 0x7f9a3806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a581f7f50 tx=0x7f9a4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.274+0000 7f54a4c00640 1 -- 172.21.15.49:0/2476728807 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5490040070 con 0x7f54a80b63d0 2024-04-15T15:18:53.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.279+0000 7fd968200640 1 Processor -- start 2024-04-15T15:18:53.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.279+0000 7fd968200640 1 -- start start 2024-04-15T15:18:53.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.281+0000 7fd968200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd96015ec40 0x7fd9600d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.281+0000 7fd968200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 0x7fd9600ba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.281+0000 7fd968200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd9600d1300 0x7fd9600d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.282+0000 7f9a54c00640 1 -- 172.21.15.49:0/3926901334 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a580b78f0 con 0x7f9a5815eab0 2024-04-15T15:18:53.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.282+0000 7fd965800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd96015ec40 0x7fd9600d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.282+0000 7fd965800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd96015ec40 0x7fd9600d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58596/0 (socket says 172.21.15.49:58596) 2024-04-15T15:18:53.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.282+0000 7fd965800640 1 -- 172.21.15.49:0/872964477 learned_addr learned my addr 172.21.15.49:0/872964477 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.282+0000 7fd968200640 1 -- 172.21.15.49:0/872964477 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd9600d3090 con 0x7fd96015ec40 2024-04-15T15:18:53.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.282+0000 7fd968200640 1 -- 172.21.15.49:0/872964477 --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fd9600ba8d0 con 0x7fd9600d1300 2024-04-15T15:18:53.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.282+0000 7fd968200640 1 -- 172.21.15.49:0/872964477 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd9600d32c0 con 0x7fd9600d2c30 2024-04-15T15:18:53.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.282+0000 7fd966200640 1 --2- 172.21.15.49:0/872964477 >> v2:172.21.15.107:3300/0 conn(0x7fd9600d1300 0x7fd9600d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.283+0000 7fd965800640 1 -- 172.21.15.49:0/872964477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 msgr2=0x7fd9600ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:53.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.283+0000 7fd965800640 1 --2- 172.21.15.49:0/872964477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 0x7fd9600ba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.283+0000 7fd965800640 1 -- 172.21.15.49:0/872964477 >> v2:172.21.15.107:3300/0 conn(0x7fd9600d1300 msgr2=0x7fd9600d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:53.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.283+0000 7fd965800640 1 --2- 172.21.15.49:0/872964477 >> v2:172.21.15.107:3300/0 conn(0x7fd9600d1300 0x7fd9600d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.283+0000 7fd965800640 1 -- 172.21.15.49:0/872964477 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9600d35a0 con 0x7fd96015ec40 2024-04-15T15:18:53.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.285+0000 7fd966200640 1 --2- 172.21.15.49:0/872964477 >> v2:172.21.15.107:3300/0 conn(0x7fd9600d1300 0x7fd9600d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:18:53.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.286+0000 7fd965800640 1 --2- 172.21.15.49:0/872964477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd96015ec40 0x7fd9600d4360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd96008dde0 tx=0x7fd94c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d67e9194c8e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.286+0000 7fd95fe00640 1 -- 172.21.15.49:0/872964477 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c034070 con 0x7fd96015ec40 2024-04-15T15:18:53.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.287+0000 7fd95fe00640 1 -- 172.21.15.49:0/872964477 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94c030610 con 0x7fd96015ec40 2024-04-15T15:18:53.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.287+0000 7fd95fe00640 1 -- 172.21.15.49:0/872964477 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c030a20 con 0x7fd96015ec40 2024-04-15T15:18:53.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.288+0000 7fd968200640 1 -- 172.21.15.49:0/872964477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd96015ec40 msgr2=0x7fd9600d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.288+0000 7fd968200640 1 --2- 172.21.15.49:0/872964477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd96015ec40 0x7fd9600d4360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd96008dde0 tx=0x7fd94c02b960 comp rx=0 tx=0).stop 2024-04-15T15:18:53.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.289+0000 7fd968200640 1 -- 172.21.15.49:0/872964477 shutdown_connections 2024-04-15T15:18:53.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.289+0000 7fd968200640 1 --2- 172.21.15.49:0/872964477 >> v2:172.21.15.107:3300/0 conn(0x7fd9600d1300 0x7fd9600d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.289+0000 7fd968200640 1 --2- 172.21.15.49:0/872964477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 0x7fd9600ba390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.289+0000 7fd968200640 1 --2- 172.21.15.49:0/872964477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd96015ec40 0x7fd9600d4360 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.289+0000 7fd968200640 1 -- 172.21.15.49:0/872964477 >> 172.21.15.49:0/872964477 conn(0x7fd960073690 msgr2=0x7fd96006cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.290+0000 7fd968200640 1 -- 172.21.15.49:0/872964477 shutdown_connections 2024-04-15T15:18:53.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.290+0000 7fd968200640 1 -- 172.21.15.49:0/872964477 wait complete. 2024-04-15T15:18:53.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.294+0000 7fd968200640 1 Processor -- start 2024-04-15T15:18:53.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.294+0000 7fd968200640 1 -- start start 2024-04-15T15:18:53.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.296+0000 7fd968200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9600d1300 0x7fd960153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.297+0000 7fd968200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 0x7fd96015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.297+0000 7fd968200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd96015ec40 0x7fd960160f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.297+0000 7fd968200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9600d3e40 con 0x7fd96015ec40 2024-04-15T15:18:53.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.297+0000 7fd968200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9600d3b60 con 0x7fd9600d1300 2024-04-15T15:18:53.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.297+0000 7fd965800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9600d1300 0x7fd960153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.297+0000 7fd964e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 0x7fd96015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.298+0000 7fd964e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 0x7fd96015fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46212/0 (socket says 172.21.15.49:46212) 2024-04-15T15:18:53.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.298+0000 7fd964e00640 1 -- 172.21.15.49:0/623940829 learned_addr learned my addr 172.21.15.49:0/623940829 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:53.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.298+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd9600b63b0 con 0x7fd9600d2c30 2024-04-15T15:18:53.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.298+0000 7fd966200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd96015ec40 0x7fd960160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.299+0000 7fd964e00640 1 -- 172.21.15.49:0/623940829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9600d1300 msgr2=0x7fd960153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.299+0000 7fd964e00640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9600d1300 0x7fd960153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.299+0000 7fd964e00640 1 -- 172.21.15.49:0/623940829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd96015ec40 msgr2=0x7fd960160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.300+0000 7fd964e00640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd96015ec40 0x7fd960160f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.300+0000 7fd964e00640 1 -- 172.21.15.49:0/623940829 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9600cd0e0 con 0x7fd9600d2c30 2024-04-15T15:18:53.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.300+0000 7fd966200640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd96015ec40 0x7fd960160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:18:53.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.301+0000 7fd964e00640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 0x7fd96015fa30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd95400b4a0 tx=0x7fd95400b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.301+0000 7fd95d600640 1 -- 172.21.15.49:0/623940829 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954015020 con 0x7fd9600d2c30 2024-04-15T15:18:53.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.301+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96015ff70 con 0x7fd9600d2c30 2024-04-15T15:18:53.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.302+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd960152be0 con 0x7fd9600d2c30 2024-04-15T15:18:53.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.302+0000 7fd95d600640 1 -- 172.21.15.49:0/623940829 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd954011830 con 0x7fd9600d2c30 2024-04-15T15:18:53.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.302+0000 7fd95d600640 1 -- 172.21.15.49:0/623940829 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954011c20 con 0x7fd9600d2c30 2024-04-15T15:18:53.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.303+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93c005290 con 0x7fd9600d2c30 2024-04-15T15:18:53.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.306+0000 7fd95d600640 1 -- 172.21.15.49:0/623940829 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd954018020 con 0x7fd9600d2c30 2024-04-15T15:18:53.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.307+0000 7fd95d600640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd94006c8f0 0x7fd94006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:53.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.307+0000 7fd95d600640 1 -- 172.21.15.49:0/623940829 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd95408b970 con 0x7fd9600d2c30 2024-04-15T15:18:53.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.309+0000 7fd965800640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd94006c8f0 0x7fd94006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:53.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.310+0000 7fd965800640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd94006c8f0 0x7fd94006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd94c036000 tx=0x7fd94c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:53.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.321+0000 7fd95d600640 1 -- 172.21.15.49:0/623940829 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd95401f020 con 0x7fd9600d2c30 2024-04-15T15:18:53.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.684+0000 7f199ce00640 1 -- 172.21.15.49:0/3805218984 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f19980d09e0 con 0x7f1998163f90 2024-04-15T15:18:53.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.686+0000 7f198e000640 1 -- 172.21.15.49:0/3805218984 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f19980d09e0 con 0x7f1998163f90 2024-04-15T15:18:53.690 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2024-04-15T15:18:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.689+0000 7f1987400640 1 -- 172.21.15.49:0/3805218984 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f197406c9f0 msgr2=0x7f197406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.689+0000 7f1987400640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f197406c9f0 0x7f197406eeb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1980006fd0 tx=0x7f1980008040 comp rx=0 tx=0).stop 2024-04-15T15:18:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.689+0000 7f1987400640 1 -- 172.21.15.49:0/3805218984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 msgr2=0x7f19980cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.689+0000 7f1987400640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 0x7f19980cae50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1988033000 tx=0x7f1988004a40 comp rx=0 tx=0).stop 2024-04-15T15:18:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.690+0000 7f1987400640 1 -- 172.21.15.49:0/3805218984 shutdown_connections 2024-04-15T15:18:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.690+0000 7f1987400640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f197406c9f0 0x7f197406eeb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.690+0000 7f1987400640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1998163f90 0x7f19980cae50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.690+0000 7f1987400640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19981669d0 0x7f1998164d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.690+0000 7f1987400640 1 --2- 172.21.15.49:0/3805218984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19980cf450 0x7f19980ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.690+0000 7f1987400640 1 -- 172.21.15.49:0/3805218984 >> 172.21.15.49:0/3805218984 conn(0x7f19980d1970 msgr2=0x7f199815b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.691+0000 7f1987400640 1 -- 172.21.15.49:0/3805218984 shutdown_connections 2024-04-15T15:18:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.691+0000 7f1987400640 1 -- 172.21.15.49:0/3805218984 wait complete. 2024-04-15T15:18:53.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.695+0000 7f7651000640 1 -- 172.21.15.49:0/404977601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f764c068c80 con 0x7f764c0ceaa0 2024-04-15T15:18:53.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.696+0000 7f7642000640 1 -- 172.21.15.49:0/404977601 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7634059820 con 0x7f764c0ceaa0 2024-04-15T15:18:53.701 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2024-04-15T15:18:53.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.700+0000 7f7651000640 1 -- 172.21.15.49:0/404977601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f762806cae0 msgr2=0x7f762806efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.700+0000 7f7651000640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f762806cae0 0x7f762806efa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f763c035000 tx=0x7f763c02e000 comp rx=0 tx=0).stop 2024-04-15T15:18:53.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.701+0000 7f7651000640 1 -- 172.21.15.49:0/404977601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764c0ceaa0 msgr2=0x7f764c1fe6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.701+0000 7f7651000640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764c0ceaa0 0x7f764c1fe6f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f763400eb70 tx=0x7f763400c520 comp rx=0 tx=0).stop 2024-04-15T15:18:53.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.701+0000 7f7651000640 1 -- 172.21.15.49:0/404977601 shutdown_connections 2024-04-15T15:18:53.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.701+0000 7f7651000640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f762806cae0 0x7f762806efa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.701+0000 7f7651000640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f764c15a910 0x7f764c1529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.701+0000 7f7651000640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764c0ceaa0 0x7f764c1fe6f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.701+0000 7f7651000640 1 --2- 172.21.15.49:0/404977601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f764c06f8a0 0x7f764c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.701+0000 7f7651000640 1 -- 172.21.15.49:0/404977601 >> 172.21.15.49:0/404977601 conn(0x7f764c0b28b0 msgr2=0x7f764c0cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.702+0000 7f7651000640 1 -- 172.21.15.49:0/404977601 shutdown_connections 2024-04-15T15:18:53.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.702+0000 7f7651000640 1 -- 172.21.15.49:0/404977601 wait complete. 2024-04-15T15:18:53.705 INFO:teuthology.orchestra.run.smithi049.stdout:60129542148 2024-04-15T15:18:53.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.703+0000 7f0f91200640 1 -- 172.21.15.49:0/3914071520 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0f64005580 con 0x7f0f8c15a910 2024-04-15T15:18:53.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.703+0000 7f0f82000640 1 -- 172.21.15.49:0/3914071520 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0f8407bae0 con 0x7f0f8c15a910 2024-04-15T15:18:53.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.707+0000 7f0f91200640 1 -- 172.21.15.49:0/3914071520 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0f6c06caf0 msgr2=0x7f0f6c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.707+0000 7f0f91200640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0f6c06caf0 0x7f0f6c06efb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c000fb0 tx=0x7f0f7c007c60 comp rx=0 tx=0).stop 2024-04-15T15:18:53.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.707+0000 7f0f91200640 1 -- 172.21.15.49:0/3914071520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15a910 msgr2=0x7f0f8c1fb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.708+0000 7fd6b9000640 1 -- 172.21.15.49:0/2731535458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd68c005850 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.707+0000 7f0f91200640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15a910 0x7f0f8c1fb6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009eb0 tx=0x7f0f84004a40 comp rx=0 tx=0).stop 2024-04-15T15:18:53.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.709+0000 7f0f91200640 1 -- 172.21.15.49:0/3914071520 shutdown_connections 2024-04-15T15:18:53.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.709+0000 7f0f91200640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0f6c06caf0 0x7f0f6c06efb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.709+0000 7f0f91200640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15a910 0x7f0f8c1fb6c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.709+0000 7f0f91200640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f8c0ceaa0 0x7f0f8c155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.709+0000 7f0f91200640 1 --2- 172.21.15.49:0/3914071520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f8c070640 0x7f0f8c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.709+0000 7f0f91200640 1 -- 172.21.15.49:0/3914071520 >> 172.21.15.49:0/3914071520 conn(0x7f0f8c0b28b0 msgr2=0x7f0f8c0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.709+0000 7f0f91200640 1 -- 172.21.15.49:0/3914071520 shutdown_connections 2024-04-15T15:18:53.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.709+0000 7f0f91200640 1 -- 172.21.15.49:0/3914071520 wait complete. 2024-04-15T15:18:53.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.709+0000 7fd6aa000640 1 -- 172.21.15.49:0/2731535458 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd6ac059ff0 con 0x7fd6b40ceaa0 2024-04-15T15:18:53.713 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2024-04-15T15:18:53.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.714+0000 7fd6b9000640 1 -- 172.21.15.49:0/2731535458 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd69006c920 msgr2=0x7fd69006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.714+0000 7fd6b9000640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd69006c920 0x7fd69006ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009ac0 tx=0x7fd6a4008040 comp rx=0 tx=0).stop 2024-04-15T15:18:53.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.714+0000 7fd6b9000640 1 -- 172.21.15.49:0/2731535458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6b40ceaa0 msgr2=0x7fd6b4153af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.714+0000 7fd6b9000640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6b40ceaa0 0x7fd6b4153af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00c770 tx=0x7fd6ac00cc30 comp rx=0 tx=0).stop 2024-04-15T15:18:53.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.714+0000 7fd6b9000640 1 -- 172.21.15.49:0/2731535458 shutdown_connections 2024-04-15T15:18:53.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.714+0000 7fd6b9000640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd69006c920 0x7fd69006ede0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.714+0000 7fd6b9000640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6b415a910 0x7fd6b4154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.714+0000 7fd6b9000640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6b40ceaa0 0x7fd6b4153af0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.714+0000 7fd6b9000640 1 --2- 172.21.15.49:0/2731535458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6b4070640 0x7fd6b41ed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.714+0000 7fd6b9000640 1 -- 172.21.15.49:0/2731535458 >> 172.21.15.49:0/2731535458 conn(0x7fd6b40b28b0 msgr2=0x7fd6b40b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.715+0000 7fd6b9000640 1 -- 172.21.15.49:0/2731535458 shutdown_connections 2024-04-15T15:18:53.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.715+0000 7fd6b9000640 1 -- 172.21.15.49:0/2731535458 wait complete. 2024-04-15T15:18:53.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2024-04-15T15:18:53.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.725+0000 7f97f6e00640 1 -- 172.21.15.49:0/636235461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f97f0073170 con 0x7f97f0072100 2024-04-15T15:18:53.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.726+0000 7f97e7e00640 1 -- 172.21.15.49:0/636235461 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f97e000b040 con 0x7f97f0072100 2024-04-15T15:18:53.728 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2024-04-15T15:18:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 -- 172.21.15.49:0/636235461 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97cc06c9c0 msgr2=0x7f97cc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97cc06c9c0 0x7f97cc06ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97d8036000 tx=0x7f97d803c040 comp rx=0 tx=0).stop 2024-04-15T15:18:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 -- 172.21.15.49:0/636235461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f0072100 msgr2=0x7f97f0153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f0072100 0x7f97f0153dc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f97e000dad0 tx=0x7f97e000df90 comp rx=0 tx=0).stop 2024-04-15T15:18:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 -- 172.21.15.49:0/636235461 shutdown_connections 2024-04-15T15:18:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97cc06c9c0 0x7f97cc06ee80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f00ceaa0 0x7f97f01514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f0072100 0x7f97f0153dc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 --2- 172.21.15.49:0/636235461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f006f8a0 0x7f97f0150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 -- 172.21.15.49:0/636235461 >> 172.21.15.49:0/636235461 conn(0x7f97f00b28b0 msgr2=0x7f97f0159c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.0 2024-04-15T15:18:53.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 -- 172.21.15.49:0/636235461 shutdown_connections 2024-04-15T15:18:53.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.730+0000 7f97f6e00640 1 -- 172.21.15.49:0/636235461 wait complete. 2024-04-15T15:18:53.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.732+0000 7f54aee00640 1 -- 172.21.15.49:0/2476728807 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f54a80d3750 con 0x7f54a80b63d0 2024-04-15T15:18:53.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.733+0000 7f54a4c00640 1 -- 172.21.15.49:0/2476728807 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f549007c080 con 0x7f54a80b63d0 2024-04-15T15:18:53.735 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2024-04-15T15:18:53.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 -- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f548406c9c0 msgr2=0x7f548406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f548406c9c0 0x7f548406ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f54a8073030 tx=0x7f5498009190 comp rx=0 tx=0).stop 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 -- 172.21.15.49:0/2476728807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a80b63d0 msgr2=0x7f54a81fd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a80b63d0 0x7f54a81fd020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5490033000 tx=0x7f54900046c0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 -- 172.21.15.49:0/2476728807 shutdown_connections 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f548406c9c0 0x7f548406ee80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54a815ec60 0x7f54a80c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54a80d3c90 0x7f54a80caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 --2- 172.21.15.49:0/2476728807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54a80b63d0 0x7f54a81fd020 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 -- 172.21.15.49:0/2476728807 >> 172.21.15.49:0/2476728807 conn(0x7f54a8073480 msgr2=0x7f54a80b82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.737+0000 7f54aee00640 1 -- 172.21.15.49:0/2476728807 shutdown_connections 2024-04-15T15:18:53.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.738+0000 7f54aee00640 1 -- 172.21.15.49:0/2476728807 wait complete. 2024-04-15T15:18:53.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.738+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9a580d2460 con 0x7f9a5815eab0 2024-04-15T15:18:53.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.738+0000 7f9a54c00640 1 -- 172.21.15.49:0/3926901334 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9a50059c30 con 0x7f9a5815eab0 2024-04-15T15:18:53.740 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2024-04-15T15:18:53.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9a3806c8f0 msgr2=0x7f9a3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9a3806c8f0 0x7f9a3806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a581f7f50 tx=0x7f9a4c009040 comp rx=0 tx=0).stop 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5815eab0 msgr2=0x7f9a580b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5815eab0 0x7f9a580b8c50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000caa0 tx=0x7f9a5000cf60 comp rx=0 tx=0).stop 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 shutdown_connections 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9a3806c8f0 0x7f9a3806edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a5815eab0 0x7f9a580b8c50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a580d32e0 0x7f9a5814f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 --2- 172.21.15.49:0/3926901334 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a580d1550 0x7f9a581529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 >> 172.21.15.49:0/3926901334 conn(0x7f9a58073240 msgr2=0x7f9a580cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 shutdown_connections 2024-04-15T15:18:53.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.742+0000 7f9a5e400640 1 -- 172.21.15.49:0/3926901334 wait complete. 2024-04-15T15:18:53.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.747+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd93c005580 con 0x7fd9600d2c30 2024-04-15T15:18:53.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.748+0000 7fd95d600640 1 -- 172.21.15.49:0/623940829 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd954059d90 con 0x7fd9600d2c30 2024-04-15T15:18:53.750 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2024-04-15T15:18:53.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.751+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd94006c8f0 msgr2=0x7fd94006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.751+0000 7fd968200640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd94006c8f0 0x7fd94006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd94c036000 tx=0x7fd94c03b040 comp rx=0 tx=0).stop 2024-04-15T15:18:53.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.751+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 msgr2=0x7fd96015fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:53.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.751+0000 7fd968200640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 0x7fd96015fa30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd95400b4a0 tx=0x7fd95400b8a0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.751+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 shutdown_connections 2024-04-15T15:18:53.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.751+0000 7fd968200640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd94006c8f0 0x7fd94006edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.751+0000 7fd968200640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9600d2c30 0x7fd96015fa30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.751+0000 7fd968200640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd96015ec40 0x7fd960160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.751+0000 7fd968200640 1 --2- 172.21.15.49:0/623940829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9600d1300 0x7fd960153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:53.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.751+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 >> 172.21.15.49:0/623940829 conn(0x7fd960073690 msgr2=0x7fd9600b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:53.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.752+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 shutdown_connections 2024-04-15T15:18:53.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:53.752+0000 7fd968200640 1 -- 172.21.15.49:0/623940829 wait complete. 2024-04-15T15:18:53.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2024-04-15T15:18:53.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.6 2024-04-15T15:18:53.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2024-04-15T15:18:53.772 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.4 2024-04-15T15:18:54.722 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-15T15:18:54.734 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-15T15:18:54.738 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-15T15:18:54.743 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-15T15:18:54.759 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-15T15:18:54.760 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-15T15:18:54.764 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-15T15:18:54.772 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-15T15:18:55.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.032+0000 7f3d73c00640 1 Processor -- start 2024-04-15T15:18:55.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.032+0000 7f3d73c00640 1 -- start start 2024-04-15T15:18:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.034+0000 7f3d73c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3d6c147fe0 0x7f3d6c06e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.034+0000 7f3d73c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c1483c0 0x7f3d6c071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.034+0000 7f3d73c00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6c0af1d0 con 0x7f3d6c0aec60 2024-04-15T15:18:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.034+0000 7f3d73c00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c06ec00 con 0x7f3d6c147fe0 2024-04-15T15:18:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.034+0000 7f3d73c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c0af4b0 con 0x7f3d6c1483c0 2024-04-15T15:18:55.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.035+0000 7f3d71c00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3d6c0aec60 0x7f3d6c162460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42862/0 (socket says 172.21.15.49:42862) 2024-04-15T15:18:55.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.035+0000 7f3d71c00640 1 -- 172.21.15.49:0/2033096178 learned_addr learned my addr 172.21.15.49:0/2033096178 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:55.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.035+0000 7f3d6be00640 1 --2- 172.21.15.49:0/2033096178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c1483c0 0x7f3d6c071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.035+0000 7f3d71200640 1 --2- 172.21.15.49:0/2033096178 >> v2:172.21.15.107:3300/0 conn(0x7f3d6c147fe0 0x7f3d6c06e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.036+0000 7f3d6b400640 1 -- 172.21.15.49:0/2033096178 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1008120292 0 0) 0x7f3d6c0af1d0 con 0x7f3d6c0aec60 2024-04-15T15:18:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.036+0000 7f3d6b400640 1 -- 172.21.15.49:0/2033096178 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d50003520 con 0x7f3d6c0aec60 2024-04-15T15:18:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.036+0000 7f3d71200640 1 -- 172.21.15.49:0/2033096178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c1483c0 msgr2=0x7f3d6c071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.036+0000 7f3d71200640 1 --2- 172.21.15.49:0/2033096178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c1483c0 0x7f3d6c071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.036+0000 7f3d71200640 1 -- 172.21.15.49:0/2033096178 >> v1:172.21.15.49:6789/0 conn(0x7f3d6c0aec60 legacy=0x7f3d6c162460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.036+0000 7f3d71200640 1 -- 172.21.15.49:0/2033096178 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c0af790 con 0x7f3d6c147fe0 2024-04-15T15:18:55.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.039+0000 7f3d71200640 1 --2- 172.21.15.49:0/2033096178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6c147fe0 0x7f3d6c06e6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3d580097a0 tx=0x7f3d5802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1729b90a0bbff44d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.040+0000 7f3d6b400640 1 -- 172.21.15.49:0/2033096178 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58031780 con 0x7f3d6c147fe0 2024-04-15T15:18:55.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.040+0000 7f3d6b400640 1 -- 172.21.15.49:0/2033096178 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d58037070 con 0x7f3d6c147fe0 2024-04-15T15:18:55.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.040+0000 7f3d6b400640 1 -- 172.21.15.49:0/2033096178 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58031b40 con 0x7f3d6c147fe0 2024-04-15T15:18:55.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.043+0000 7f3d73c00640 1 -- 172.21.15.49:0/2033096178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6c147fe0 msgr2=0x7f3d6c06e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.043+0000 7f3d73c00640 1 --2- 172.21.15.49:0/2033096178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6c147fe0 0x7f3d6c06e6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3d580097a0 tx=0x7f3d5802b820 comp rx=0 tx=0).stop 2024-04-15T15:18:55.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.043+0000 7f3d73c00640 1 -- 172.21.15.49:0/2033096178 shutdown_connections 2024-04-15T15:18:55.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.044+0000 7f3d73c00640 1 --2- 172.21.15.49:0/2033096178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c1483c0 0x7f3d6c071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.044+0000 7f3d73c00640 1 --2- 172.21.15.49:0/2033096178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6c147fe0 0x7f3d6c06e6c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.044+0000 7f3d73c00640 1 -- 172.21.15.49:0/2033096178 >> 172.21.15.49:0/2033096178 conn(0x7f3d6c067b70 msgr2=0x7f3d6c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.044+0000 7f3d73c00640 1 -- 172.21.15.49:0/2033096178 shutdown_connections 2024-04-15T15:18:55.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.044+0000 7f3d73c00640 1 -- 172.21.15.49:0/2033096178 wait complete. 2024-04-15T15:18:55.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.047+0000 7f3d73c00640 1 Processor -- start 2024-04-15T15:18:55.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.047+0000 7f3d73c00640 1 -- start start 2024-04-15T15:18:55.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.049+0000 7f3d73c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6c0aec60 0x7f3d6c146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.050+0000 7f3d73c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c147fe0 0x7f3d6c145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.050+0000 7f3d73c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d6c1483c0 0x7f3d6c1448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.051+0000 7f3d73c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c06f0b0 con 0x7f3d6c1483c0 2024-04-15T15:18:55.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.051+0000 7f3d73c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c0afd50 con 0x7f3d6c0aec60 2024-04-15T15:18:55.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.051+0000 7f3d73c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c0b4660 con 0x7f3d6c147fe0 2024-04-15T15:18:55.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.051+0000 7f3d6be00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c147fe0 0x7f3d6c145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.051+0000 7f3d6be00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c147fe0 0x7f3d6c145000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:43096/0 (socket says 172.21.15.49:43096) 2024-04-15T15:18:55.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.051+0000 7f3d6be00640 1 -- 172.21.15.49:0/332411782 learned_addr learned my addr 172.21.15.49:0/332411782 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.051+0000 7f3d71200640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6c0aec60 0x7f3d6c146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.052+0000 7f3d6be00640 1 -- 172.21.15.49:0/332411782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6c0aec60 msgr2=0x7f3d6c146780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.052+0000 7f3d6be00640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6c0aec60 0x7f3d6c146780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.052+0000 7f3d6be00640 1 -- 172.21.15.49:0/332411782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d6c1483c0 msgr2=0x7f3d6c1448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.052+0000 7f3d6be00640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d6c1483c0 0x7f3d6c1448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.052+0000 7f3d6be00640 1 -- 172.21.15.49:0/332411782 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c1456e0 con 0x7f3d6c147fe0 2024-04-15T15:18:55.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.053+0000 7f3d71200640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6c0aec60 0x7f3d6c146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:55.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.053+0000 7f3d6be00640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c147fe0 0x7f3d6c145000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00e870 tx=0x7f3d5c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.054+0000 7f3d68c00640 1 -- 172.21.15.49:0/332411782 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c017070 con 0x7f3d6c147fe0 2024-04-15T15:18:55.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.054+0000 7f3d68c00640 1 -- 172.21.15.49:0/332411782 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c00c940 con 0x7f3d6c147fe0 2024-04-15T15:18:55.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.054+0000 7f3d68c00640 1 -- 172.21.15.49:0/332411782 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c00cd30 con 0x7f3d6c147fe0 2024-04-15T15:18:55.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.054+0000 7f3d73c00640 1 -- 172.21.15.49:0/332411782 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c1459c0 con 0x7f3d6c147fe0 2024-04-15T15:18:55.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.055+0000 7f3d73c00640 1 -- 172.21.15.49:0/332411782 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c145f80 con 0x7f3d6c147fe0 2024-04-15T15:18:55.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.059+0000 7f3d62000640 1 -- 172.21.15.49:0/332411782 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c0755c0 con 0x7f3d6c147fe0 2024-04-15T15:18:55.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.059+0000 7f3d68c00640 1 -- 172.21.15.49:0/332411782 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d5c01a070 con 0x7f3d6c147fe0 2024-04-15T15:18:55.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.060+0000 7f3d68c00640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3d4c06c920 0x7f3d4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.061+0000 7f3d68c00640 1 -- 172.21.15.49:0/332411782 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3d5c08be10 con 0x7f3d6c147fe0 2024-04-15T15:18:55.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.062+0000 7f3d71200640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3d4c06c920 0x7f3d4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.063+0000 7f3d71200640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3d4c06c920 0x7f3d4c06ede0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d58036000 tx=0x7f3d5803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.076+0000 7f3d68c00640 1 -- 172.21.15.49:0/332411782 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d5c05a1a0 con 0x7f3d6c147fe0 2024-04-15T15:18:55.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.081+0000 7f9b36400640 1 Processor -- start 2024-04-15T15:18:55.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.081+0000 7f9b36400640 1 -- start start 2024-04-15T15:18:55.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.082+0000 7f2717e00640 1 Processor -- start 2024-04-15T15:18:55.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.083+0000 7f9b36400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b280e3160 0x7f9b280e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.083+0000 7f9b36400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b28058810 con 0x7f9b28053cb0 2024-04-15T15:18:55.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.083+0000 7f9b36400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b28058af0 con 0x7f9b2803c0a0 2024-04-15T15:18:55.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.083+0000 7f9b36400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9b28007b70 con 0x7f9b280e3160 2024-04-15T15:18:55.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.084+0000 7f9b35000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b280e3160 0x7f9b280e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.084+0000 7f9b35000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b280e3160 0x7f9b280e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:43106/0 (socket says 172.21.15.49:43106) 2024-04-15T15:18:55.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.084+0000 7f9b35000640 1 -- 172.21.15.49:0/650417704 learned_addr learned my addr 172.21.15.49:0/650417704 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.082+0000 7f2717e00640 1 -- start start 2024-04-15T15:18:55.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.084+0000 7f2717e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f27100069e0 0x7f2710006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.084+0000 7f2717e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2710053720 con 0x7f2710005e90 2024-04-15T15:18:55.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.084+0000 7f2717e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2710053a00 con 0x7f27100ead00 2024-04-15T15:18:55.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.084+0000 7f2717e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f2710054330 con 0x7f27100069e0 2024-04-15T15:18:55.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.085+0000 7f9b35000640 1 -- 172.21.15.49:0/650417704 >> v1:172.21.15.107:6789/0 conn(0x7f9b2803c0a0 legacy=0x7f9b28058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.085+0000 7f9b35000640 1 -- 172.21.15.49:0/650417704 >> v1:172.21.15.49:6789/0 conn(0x7f9b28053cb0 legacy=0x7f9b28054120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.085+0000 7f2716a00640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f27100ead00 0x7f271004fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:46318/0 (socket says 172.21.15.49:46318) 2024-04-15T15:18:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.085+0000 7f9b35000640 1 -- 172.21.15.49:0/650417704 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b28006360 con 0x7f9b280e3160 2024-04-15T15:18:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.085+0000 7f2716000640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2710005e90 0x7f27100062d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42890/0 (socket says 172.21.15.49:42890) 2024-04-15T15:18:55.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.085+0000 7f2716000640 1 -- 172.21.15.49:0/2823388878 learned_addr learned my addr 172.21.15.49:0/2823388878 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:55.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.086+0000 7f2717400640 1 --2- 172.21.15.49:0/2823388878 >> v2:172.21.15.49:3300/0 conn(0x7f27100069e0 0x7f2710006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.086+0000 7f2715600640 1 -- 172.21.15.49:0/2823388878 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1910111657 0 0) 0x7f2710053a00 con 0x7f27100ead00 2024-04-15T15:18:55.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.086+0000 7f2715600640 1 -- 172.21.15.49:0/2823388878 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26fc003520 con 0x7f27100ead00 2024-04-15T15:18:55.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.087+0000 7f2715600640 1 -- 172.21.15.49:0/2823388878 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596868864 0 0) 0x7f2710053720 con 0x7f2710005e90 2024-04-15T15:18:55.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.087+0000 7f2715600640 1 -- 172.21.15.49:0/2823388878 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2710053a00 con 0x7f2710005e90 2024-04-15T15:18:55.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.087+0000 7f9b35000640 1 --2- 172.21.15.49:0/650417704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b280e3160 0x7f9b280e7780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009820 tx=0x7f9b2402b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d388ea623963a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.087+0000 7f2717400640 1 -- 172.21.15.49:0/2823388878 >> v1:172.21.15.107:6789/0 conn(0x7f27100ead00 legacy=0x7f271004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.087+0000 7f2717400640 1 -- 172.21.15.49:0/2823388878 >> v1:172.21.15.49:6789/0 conn(0x7f2710005e90 legacy=0x7f27100062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.088+0000 7f9b2f400640 1 -- 172.21.15.49:0/650417704 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24037070 con 0x7f9b280e3160 2024-04-15T15:18:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.088+0000 7f9b2f400640 1 -- 172.21.15.49:0/650417704 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b24030510 con 0x7f9b280e3160 2024-04-15T15:18:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.088+0000 7f9b2f400640 1 -- 172.21.15.49:0/650417704 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b240308e0 con 0x7f9b280e3160 2024-04-15T15:18:55.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.088+0000 7f2717400640 1 -- 172.21.15.49:0/2823388878 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2710053ce0 con 0x7f27100069e0 2024-04-15T15:18:55.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.089+0000 7f9b36400640 1 -- 172.21.15.49:0/650417704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b280e3160 msgr2=0x7f9b280e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.089+0000 7f9b36400640 1 --2- 172.21.15.49:0/650417704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b280e3160 0x7f9b280e7780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009820 tx=0x7f9b2402b600 comp rx=0 tx=0).stop 2024-04-15T15:18:55.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.089+0000 7f9b36400640 1 -- 172.21.15.49:0/650417704 shutdown_connections 2024-04-15T15:18:55.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.089+0000 7f9b36400640 1 --2- 172.21.15.49:0/650417704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b280e3160 0x7f9b280e7780 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.089+0000 7f9b36400640 1 -- 172.21.15.49:0/650417704 >> 172.21.15.49:0/650417704 conn(0x7f9b28058ee0 msgr2=0x7f9b280592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.090+0000 7f9b36400640 1 -- 172.21.15.49:0/650417704 shutdown_connections 2024-04-15T15:18:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.090+0000 7f9b36400640 1 -- 172.21.15.49:0/650417704 wait complete. 2024-04-15T15:18:55.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.091+0000 7f2717400640 1 --2- 172.21.15.49:0/2823388878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27100069e0 0x7f2710006e50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2718050b00 tx=0x7f27180a72d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5e047f0f7e605c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.092+0000 7f9b36400640 1 Processor -- start 2024-04-15T15:18:55.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.092+0000 7f9b36400640 1 -- start start 2024-04-15T15:18:55.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.092+0000 7fc217800640 1 Processor -- start 2024-04-15T15:18:55.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.092+0000 7f2715600640 1 -- 172.21.15.49:0/2823388878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27180b4070 con 0x7f27100069e0 2024-04-15T15:18:55.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.092+0000 7f2715600640 1 -- 172.21.15.49:0/2823388878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27180af440 con 0x7f27100069e0 2024-04-15T15:18:55.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.092+0000 7fc217800640 1 -- start start 2024-04-15T15:18:55.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.092+0000 7f2715600640 1 -- 172.21.15.49:0/2823388878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27180b4070 con 0x7f27100069e0 2024-04-15T15:18:55.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.093+0000 7f9b36400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b2803c0a0 0x7f9b28186b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.093+0000 7f9b36400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b28053cb0 0x7f9b280d92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.093+0000 7f9b36400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b280e3160 0x7f9b2803d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.093+0000 7f9b36400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b280ee050 con 0x7f9b28053cb0 2024-04-15T15:18:55.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.093+0000 7f9b36400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b280e3540 con 0x7f9b280e3160 2024-04-15T15:18:55.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.093+0000 7f9b36400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9b280ee1d0 con 0x7f9b2803c0a0 2024-04-15T15:18:55.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f9b2fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b28053cb0 0x7f9b280d92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f2717e00640 1 -- 172.21.15.49:0/2823388878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27100069e0 msgr2=0x7f2710006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7fc217800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc21015ec60 0x7fc2100ba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7fc217800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc2100b63d0 0x7fc2100b6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7fc217800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2100b7a80 0x7fc2100b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f2717e00640 1 --2- 172.21.15.49:0/2823388878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27100069e0 0x7f2710006e50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2718050b00 tx=0x7f27180a72d0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7fc217800640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc2100b6d70 con 0x7fc2100b7a80 2024-04-15T15:18:55.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7fc217800640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fc2100b8f30 con 0x7fc2100b63d0 2024-04-15T15:18:55.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7fc217800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc2100b9210 con 0x7fc21015ec60 2024-04-15T15:18:55.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f9b2fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b28053cb0 0x7f9b280d92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39782/0 (socket says 172.21.15.49:39782) 2024-04-15T15:18:55.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f9b2fe00640 1 -- 172.21.15.49:0/507279726 learned_addr learned my addr 172.21.15.49:0/507279726 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f9b35000640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b2803c0a0 0x7f9b28186b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f9b35a00640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b280e3160 0x7f9b2803d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f9b2fe00640 1 -- 172.21.15.49:0/507279726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b2803c0a0 msgr2=0x7f9b28186b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f9b2fe00640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b2803c0a0 0x7f9b28186b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f9b2fe00640 1 -- 172.21.15.49:0/507279726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b280e3160 msgr2=0x7f9b2803d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f9b2fe00640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b280e3160 0x7f9b2803d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f9b2fe00640 1 -- 172.21.15.49:0/507279726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b28187720 con 0x7f9b28053cb0 2024-04-15T15:18:55.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f2717e00640 1 -- 172.21.15.49:0/2823388878 shutdown_connections 2024-04-15T15:18:55.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f2717e00640 1 --2- 172.21.15.49:0/2823388878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27100069e0 0x7f2710006e50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.094+0000 7f2717e00640 1 -- 172.21.15.49:0/2823388878 >> 172.21.15.49:0/2823388878 conn(0x7f2710009fb0 msgr2=0x7f271000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7f9b2fe00640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b28053cb0 0x7f9b280d92f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00c8a0 tx=0x7f9b1c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7f9dacc00640 1 Processor -- start 2024-04-15T15:18:55.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7fc215800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2100b7a80 0x7fc2100b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7f2717e00640 1 -- 172.21.15.49:0/2823388878 shutdown_connections 2024-04-15T15:18:55.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7f9b2cc00640 1 -- 172.21.15.49:0/507279726 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c017070 con 0x7f9b28053cb0 2024-04-15T15:18:55.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7f9b2cc00640 1 -- 172.21.15.49:0/507279726 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1c005270 con 0x7f9b28053cb0 2024-04-15T15:18:55.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7f9b2cc00640 1 -- 172.21.15.49:0/507279726 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00a860 con 0x7f9b28053cb0 2024-04-15T15:18:55.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.096+0000 7f9b36400640 1 -- 172.21.15.49:0/507279726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b28187a00 con 0x7f9b28053cb0 2024-04-15T15:18:55.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.096+0000 7f9b36400640 1 -- 172.21.15.49:0/507279726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b28187fc0 con 0x7f9b28053cb0 2024-04-15T15:18:55.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7f2717e00640 1 -- 172.21.15.49:0/2823388878 wait complete. 2024-04-15T15:18:55.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7fc215800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2100b7a80 0x7fc2100b7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39796/0 (socket says 172.21.15.49:39796) 2024-04-15T15:18:55.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7fc215800640 1 -- 172.21.15.49:0/3383565530 learned_addr learned my addr 172.21.15.49:0/3383565530 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.095+0000 7f9dacc00640 1 -- start start 2024-04-15T15:18:55.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.097+0000 7f9dacc00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9da815ec40 0x7f9da80d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.097+0000 7f9dacc00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9da80d1300 0x7f9da80d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.097+0000 7f9dacc00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da80d32c0 con 0x7f9da80d2c30 2024-04-15T15:18:55.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.098+0000 7fc214e00640 1 --2- 172.21.15.49:0/3383565530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc21015ec60 0x7fc2100ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.098+0000 7fc20fe00640 1 --2- 172.21.15.49:0/3383565530 >> v2:172.21.15.107:3300/0 conn(0x7fc2100b63d0 0x7fc2100b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.098+0000 7fc215800640 1 -- 172.21.15.49:0/3383565530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc21015ec60 msgr2=0x7fc2100ba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:55.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.098+0000 7fc215800640 1 --2- 172.21.15.49:0/3383565530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc21015ec60 0x7fc2100ba270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.098+0000 7fc215800640 1 -- 172.21.15.49:0/3383565530 >> v2:172.21.15.107:3300/0 conn(0x7fc2100b63d0 msgr2=0x7fc2100b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:55.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.098+0000 7f9b36400640 1 -- 172.21.15.49:0/507279726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b28005390 con 0x7f9b28053cb0 2024-04-15T15:18:55.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.098+0000 7fc215800640 1 --2- 172.21.15.49:0/3383565530 >> v2:172.21.15.107:3300/0 conn(0x7fc2100b63d0 0x7fc2100b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.098+0000 7fc215800640 1 -- 172.21.15.49:0/3383565530 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2100b94f0 con 0x7fc2100b7a80 2024-04-15T15:18:55.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.099+0000 7fc20fe00640 1 --2- 172.21.15.49:0/3383565530 >> v2:172.21.15.107:3300/0 conn(0x7fc2100b63d0 0x7fc2100b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:55.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.099+0000 7f2717e00640 1 Processor -- start 2024-04-15T15:18:55.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.099+0000 7f2717e00640 1 -- start start 2024-04-15T15:18:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.097+0000 7f9dacc00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f9da80d3090 con 0x7f9da815ec40 2024-04-15T15:18:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.097+0000 7f9dacc00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f9da80d35a0 con 0x7f9da80d1300 2024-04-15T15:18:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.098+0000 7f9da5400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9da80d2c30 0x7f9da80ba390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42898/0 (socket says 172.21.15.49:42898) 2024-04-15T15:18:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.098+0000 7f9da5400640 1 -- 172.21.15.49:0/126762855 learned_addr learned my addr 172.21.15.49:0/126762855 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.101+0000 7f9da4a00640 1 -- 172.21.15.49:0/126762855 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3230495227 0 0) 0x7f9da80d32c0 con 0x7f9da80d2c30 2024-04-15T15:18:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.099+0000 7f9b2cc00640 1 -- 172.21.15.49:0/507279726 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b1c01a070 con 0x7f9b28053cb0 2024-04-15T15:18:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.101+0000 7f9b2cc00640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9b1006c970 0x7f9b1006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.101+0000 7f9da4a00640 1 -- 172.21.15.49:0/126762855 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d8c003520 con 0x7f9da80d2c30 2024-04-15T15:18:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.102+0000 7f9b35000640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9b1006c970 0x7f9b1006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.102+0000 7f2717e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2710005e90 0x7f27100d3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.101+0000 7fc215800640 1 --2- 172.21.15.49:0/3383565530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2100b7a80 0x7fc2100b7ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc2100b7510 tx=0x7fc20802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fa8dee850f62df server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.102+0000 7f2717e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27100069e0 0x7f2710048420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.102+0000 7f2717e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27100ead00 0x7f2710048960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.102+0000 7f9da5e00640 1 --2- 172.21.15.49:0/126762855 >> v2:172.21.15.49:3300/0 conn(0x7f9da815ec40 0x7f9da80d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.101+0000 7f9da6800640 1 --2- 172.21.15.49:0/126762855 >> v2:172.21.15.107:3300/0 conn(0x7f9da80d1300 0x7f9da80d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.102+0000 7f2717e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f27100547c0 con 0x7f27100ead00 2024-04-15T15:18:55.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.103+0000 7f2717e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f27100501a0 con 0x7f27100069e0 2024-04-15T15:18:55.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.103+0000 7f2716a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2710005e90 0x7f27100d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.103+0000 7f2717e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f2710054940 con 0x7f2710005e90 2024-04-15T15:18:55.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.103+0000 7f2716a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2710005e90 0x7f27100d3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:43140/0 (socket says 172.21.15.49:43140) 2024-04-15T15:18:55.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.103+0000 7f2716a00640 1 -- 172.21.15.49:0/1762315435 learned_addr learned my addr 172.21.15.49:0/1762315435 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.103+0000 7f2717400640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27100ead00 0x7f2710048960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.103+0000 7f9b2cc00640 1 -- 172.21.15.49:0/507279726 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9b1c0896e0 con 0x7f9b28053cb0 2024-04-15T15:18:55.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.103+0000 7f2717400640 1 -- 172.21.15.49:0/1762315435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2710005e90 msgr2=0x7f27100d3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f2717400640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2710005e90 0x7f27100d3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f9b35000640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9b1006c970 0x7f9b1006ee30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b24034000 tx=0x7f9b24008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f2717400640 1 -- 172.21.15.49:0/1762315435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27100069e0 msgr2=0x7f2710048420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.103+0000 7f2716000640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27100069e0 0x7f2710048420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f2717400640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27100069e0 0x7f2710048420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f2717400640 1 -- 172.21.15.49:0/1762315435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27100e4a20 con 0x7f27100ead00 2024-04-15T15:18:55.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f9da5e00640 1 -- 172.21.15.49:0/126762855 >> v2:172.21.15.107:3300/0 conn(0x7f9da80d1300 msgr2=0x7f9da80d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:55.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f9da5e00640 1 --2- 172.21.15.49:0/126762855 >> v2:172.21.15.107:3300/0 conn(0x7f9da80d1300 0x7f9da80d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f9da5e00640 1 -- 172.21.15.49:0/126762855 >> v1:172.21.15.49:6789/0 conn(0x7f9da80d2c30 legacy=0x7f9da80ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f9da5e00640 1 -- 172.21.15.49:0/126762855 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da80d3880 con 0x7f9da815ec40 2024-04-15T15:18:55.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f2716000640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27100069e0 0x7f2710048420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:55.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.104+0000 7f2717400640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27100ead00 0x7f2710048960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2718065750 tx=0x7f271806b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.105+0000 7fba9ca00640 1 Processor -- start 2024-04-15T15:18:55.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.105+0000 7f9da5e00640 1 --2- 172.21.15.49:0/126762855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da815ec40 0x7f9da80d4360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9da80cef10 tx=0x7f9d9802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216a09256b553944 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.105+0000 7fba9ca00640 1 -- start start 2024-04-15T15:18:55.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.106+0000 7f99aa800640 1 Processor -- start 2024-04-15T15:18:55.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.106+0000 7f99aa800640 1 -- start start 2024-04-15T15:18:55.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.107+0000 7f9da4a00640 1 -- 172.21.15.49:0/126762855 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98035020 con 0x7f9da815ec40 2024-04-15T15:18:55.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.107+0000 7f9da4a00640 1 -- 172.21.15.49:0/126762855 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98031870 con 0x7f9da815ec40 2024-04-15T15:18:55.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.107+0000 7f9da4a00640 1 -- 172.21.15.49:0/126762855 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98031c80 con 0x7f9da815ec40 2024-04-15T15:18:55.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.107+0000 7fc20f400640 1 -- 172.21.15.49:0/3383565530 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20802ca80 con 0x7fc2100b7a80 2024-04-15T15:18:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.107+0000 7f9b2cc00640 1 -- 172.21.15.49:0/507279726 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b1c0594d0 con 0x7f9b28053cb0 2024-04-15T15:18:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.107+0000 7fba9ca00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fba98162dd0 0x7fba980afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.108+0000 7fc20f400640 1 -- 172.21.15.49:0/3383565530 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc20802cc20 con 0x7fc2100b7a80 2024-04-15T15:18:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.108+0000 7fc20f400640 1 -- 172.21.15.49:0/3383565530 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080343f0 con 0x7fc2100b7a80 2024-04-15T15:18:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.108+0000 7f270ea00640 1 -- 172.21.15.49:0/1762315435 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27180b4040 con 0x7f27100ead00 2024-04-15T15:18:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.108+0000 7fba9ca00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba980ce760 con 0x7fba980b5530 2024-04-15T15:18:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.108+0000 7f2717e00640 1 -- 172.21.15.49:0/1762315435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271004cbe0 con 0x7f27100ead00 2024-04-15T15:18:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.108+0000 7f2717e00640 1 -- 172.21.15.49:0/1762315435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271004b240 con 0x7f27100ead00 2024-04-15T15:18:55.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.108+0000 7fba9ca00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba980cea40 con 0x7fba980b5b00 2024-04-15T15:18:55.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.108+0000 7fba9ca00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fba980ced20 con 0x7fba98162dd0 2024-04-15T15:18:55.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.108+0000 7f9dacc00640 1 -- 172.21.15.49:0/126762855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da815ec40 msgr2=0x7f9da80d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.108+0000 7f9dacc00640 1 --2- 172.21.15.49:0/126762855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da815ec40 0x7f9da80d4360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9da80cef10 tx=0x7f9d9802b910 comp rx=0 tx=0).stop 2024-04-15T15:18:55.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7fc217800640 1 -- 172.21.15.49:0/3383565530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2100b7a80 msgr2=0x7fc2100b7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7fc217800640 1 --2- 172.21.15.49:0/3383565530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2100b7a80 0x7fc2100b7ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc2100b7510 tx=0x7fc20802bee0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7f99aa800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f999c0e3160 0x7f999c0e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7f270ea00640 1 -- 172.21.15.49:0/1762315435 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27180be070 con 0x7f27100ead00 2024-04-15T15:18:55.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7fc217800640 1 -- 172.21.15.49:0/3383565530 shutdown_connections 2024-04-15T15:18:55.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7fc217800640 1 --2- 172.21.15.49:0/3383565530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2100b7a80 0x7fc2100b7ed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7fc217800640 1 --2- 172.21.15.49:0/3383565530 >> v2:172.21.15.107:3300/0 conn(0x7fc2100b63d0 0x7fc2100b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7fc217800640 1 --2- 172.21.15.49:0/3383565530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc21015ec60 0x7fc2100ba270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7fc217800640 1 -- 172.21.15.49:0/3383565530 >> 172.21.15.49:0/3383565530 conn(0x7fc210073480 msgr2=0x7fc210073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7fba97400640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fba980b5b00 0x7fba980b5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:46332/0 (socket says 172.21.15.49:46332) 2024-04-15T15:18:55.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7fba97400640 1 -- 172.21.15.49:0/1113615835 learned_addr learned my addr 172.21.15.49:0/1113615835 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:55.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7fba97e00640 1 --1- 172.21.15.49:0/1113615835 >> v1:172.21.15.49:6789/0 conn(0x7fba980b5530 0x7fba980b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42910/0 (socket says 172.21.15.49:42910) 2024-04-15T15:18:55.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.110+0000 7f9dacc00640 1 -- 172.21.15.49:0/126762855 shutdown_connections 2024-04-15T15:18:55.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.110+0000 7f9dacc00640 1 --2- 172.21.15.49:0/126762855 >> v2:172.21.15.107:3300/0 conn(0x7f9da80d1300 0x7f9da80d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.110+0000 7f9dacc00640 1 --2- 172.21.15.49:0/126762855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da815ec40 0x7f9da80d4360 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.110+0000 7f9dacc00640 1 -- 172.21.15.49:0/126762855 >> 172.21.15.49:0/126762855 conn(0x7f9da8073690 msgr2=0x7f9da806cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.110+0000 7fc217800640 1 -- 172.21.15.49:0/3383565530 shutdown_connections 2024-04-15T15:18:55.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.110+0000 7fc217800640 1 -- 172.21.15.49:0/3383565530 wait complete. 2024-04-15T15:18:55.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.110+0000 7fba96a00640 1 --2- 172.21.15.49:0/1113615835 >> v2:172.21.15.107:3300/0 conn(0x7fba98162dd0 0x7fba980afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.110+0000 7f270ea00640 1 -- 172.21.15.49:0/1762315435 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27180b4040 con 0x7f27100ead00 2024-04-15T15:18:55.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7f99aa800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 0x7f999c054140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7f99aa800640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f999c054680 con 0x7f999c03c0a0 2024-04-15T15:18:55.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7f99aa800640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f999c007b70 con 0x7f999c0e3160 2024-04-15T15:18:55.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.109+0000 7f99aa800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f999c058810 con 0x7f999c053cb0 2024-04-15T15:18:55.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.111+0000 7f99a9400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f999c0e3160 0x7f999c0e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.111+0000 7f99a9400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f999c0e3160 0x7f999c0e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:58296/0 (socket says 172.21.15.49:58296) 2024-04-15T15:18:55.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.111+0000 7f99a9400640 1 -- 172.21.15.49:0/2079287095 learned_addr learned my addr 172.21.15.49:0/2079287095 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.111+0000 7f2717e00640 1 -- 172.21.15.49:0/1762315435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2710005390 con 0x7f27100ead00 2024-04-15T15:18:55.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.111+0000 7f99a9e00640 1 --2- 172.21.15.49:0/2079287095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 0x7f999c054140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.111+0000 7f99a9400640 1 -- 172.21.15.49:0/2079287095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 msgr2=0x7f999c054140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:55.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.112+0000 7f9dacc00640 1 -- 172.21.15.49:0/126762855 shutdown_connections 2024-04-15T15:18:55.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.112+0000 7f9dacc00640 1 -- 172.21.15.49:0/126762855 wait complete. 2024-04-15T15:18:55.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.111+0000 7fba96a00640 1 -- 172.21.15.49:0/1113615835 >> v1:172.21.15.107:6789/0 conn(0x7fba980b5b00 legacy=0x7fba980b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.111+0000 7fba96a00640 1 -- 172.21.15.49:0/1113615835 >> v1:172.21.15.49:6789/0 conn(0x7fba980b5530 legacy=0x7fba980b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.111+0000 7fba96a00640 1 -- 172.21.15.49:0/1113615835 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba980ce410 con 0x7fba98162dd0 2024-04-15T15:18:55.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.110+0000 7fba96000640 1 -- 172.21.15.49:0/1113615835 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1152743644 0 0) 0x7fba980ce760 con 0x7fba980b5530 2024-04-15T15:18:55.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.113+0000 7fc217800640 1 Processor -- start 2024-04-15T15:18:55.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.112+0000 7f99a9400640 1 --2- 172.21.15.49:0/2079287095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 0x7f999c054140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.112+0000 7f99a9400640 1 -- 172.21.15.49:0/2079287095 >> v1:172.21.15.49:6789/0 conn(0x7f999c03c0a0 legacy=0x7f999c058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.113+0000 7fc217800640 1 -- start start 2024-04-15T15:18:55.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.112+0000 7f99a9400640 1 -- 172.21.15.49:0/2079287095 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c058af0 con 0x7f999c0e3160 2024-04-15T15:18:55.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.113+0000 7fba96a00640 1 --2- 172.21.15.49:0/1113615835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba98162dd0 0x7fba980afdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba84009720 tx=0x7fba8402b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec05ef074323cde server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.112+0000 7f99a9e00640 1 --2- 172.21.15.49:0/2079287095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 0x7f999c054140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:55.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.114+0000 7fba96000640 1 -- 172.21.15.49:0/1113615835 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84037070 con 0x7fba98162dd0 2024-04-15T15:18:55.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.114+0000 7fba96000640 1 -- 172.21.15.49:0/1113615835 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba84030710 con 0x7fba98162dd0 2024-04-15T15:18:55.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.114+0000 7fba96000640 1 -- 172.21.15.49:0/1113615835 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84037070 con 0x7fba98162dd0 2024-04-15T15:18:55.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.115+0000 7fc217800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2100b63d0 0x7fc21014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.115+0000 7fc217800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2100b7a80 0x7fc21014f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7fc217800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc21015ec60 0x7fc210151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7fba9ca00640 1 -- 172.21.15.49:0/1113615835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba98162dd0 msgr2=0x7fba980afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7fba9ca00640 1 --2- 172.21.15.49:0/1113615835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba98162dd0 0x7fba980afdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba84009720 tx=0x7fba8402b730 comp rx=0 tx=0).stop 2024-04-15T15:18:55.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7f9dacc00640 1 Processor -- start 2024-04-15T15:18:55.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7fc214e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2100b63d0 0x7fc21014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7fc217800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc21015f040 con 0x7fc21015ec60 2024-04-15T15:18:55.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7fc217800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc2100b9ab0 con 0x7fc2100b7a80 2024-04-15T15:18:55.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7fc217800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc2100ba7b0 con 0x7fc2100b63d0 2024-04-15T15:18:55.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.114+0000 7f270ea00640 1 -- 172.21.15.49:0/1762315435 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27180b8410 con 0x7f27100ead00 2024-04-15T15:18:55.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7f270ea00640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f26f806caf0 0x7f26f806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7f270ea00640 1 -- 172.21.15.49:0/1762315435 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2718127c40 con 0x7f27100ead00 2024-04-15T15:18:55.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7fc20fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2100b7a80 0x7fc21014f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7fc20fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2100b7a80 0x7fc21014f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:58310/0 (socket says 172.21.15.49:58310) 2024-04-15T15:18:55.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7fc20fe00640 1 -- 172.21.15.49:0/2425751432 learned_addr learned my addr 172.21.15.49:0/2425751432 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.116+0000 7f9dacc00640 1 -- start start 2024-04-15T15:18:55.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.117+0000 7f2716a00640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f26f806caf0 0x7f26f806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.117+0000 7fc215800640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc21015ec60 0x7fc210151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.117+0000 7fc20fe00640 1 -- 172.21.15.49:0/2425751432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2100b63d0 msgr2=0x7fc21014e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.117+0000 7fc20fe00640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2100b63d0 0x7fc21014e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.117+0000 7fc20fe00640 1 -- 172.21.15.49:0/2425751432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc21015ec60 msgr2=0x7fc210151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.117+0000 7fc20fe00640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc21015ec60 0x7fc210151690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.117+0000 7fc20fe00640 1 -- 172.21.15.49:0/2425751432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21014e780 con 0x7fc2100b7a80 2024-04-15T15:18:55.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.117+0000 7f99a9400640 1 --2- 172.21.15.49:0/2079287095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999c0e3160 0x7f999c0e7780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9998009870 tx=0x7f999802b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=464441c1bd62551b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.118+0000 7f99a3e00640 1 -- 172.21.15.49:0/2079287095 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998037070 con 0x7f999c0e3160 2024-04-15T15:18:55.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.118+0000 7f99a3e00640 1 -- 172.21.15.49:0/2079287095 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9998030710 con 0x7f999c0e3160 2024-04-15T15:18:55.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.118+0000 7f99a3e00640 1 -- 172.21.15.49:0/2079287095 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998030ae0 con 0x7f999c0e3160 2024-04-15T15:18:55.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.118+0000 7fba9ca00640 1 -- 172.21.15.49:0/1113615835 shutdown_connections 2024-04-15T15:18:55.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.118+0000 7fba9ca00640 1 --2- 172.21.15.49:0/1113615835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba98162dd0 0x7fba980afdf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.118+0000 7fba9ca00640 1 -- 172.21.15.49:0/1113615835 >> 172.21.15.49:0/1113615835 conn(0x7fba980b23a0 msgr2=0x7fba980b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.118+0000 7f2716a00640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f26f806caf0 0x7f26f806efb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2708001a40 tx=0x7f2708001710 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.118+0000 7fc20fe00640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2100b7a80 0x7fc21014f820 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc20400c820 tx=0x7fc20400cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.118+0000 7f9dacc00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9da80d1300 0x7f9da8153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.118+0000 7f9dacc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da80d2c30 0x7f9da80cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.119+0000 7f9dacc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da815ec40 0x7f9da81615c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.119+0000 7f9da5400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da80d2c30 0x7f9da80cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.119+0000 7f9da5400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da80d2c30 0x7f9da80cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39844/0 (socket says 172.21.15.49:39844) 2024-04-15T15:18:55.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.119+0000 7f9da5400640 1 -- 172.21.15.49:0/1864501056 learned_addr learned my addr 172.21.15.49:0/1864501056 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.119+0000 7f99aa800640 1 -- 172.21.15.49:0/2079287095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999c0e3160 msgr2=0x7f999c0e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.119+0000 7f99aa800640 1 --2- 172.21.15.49:0/2079287095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999c0e3160 0x7f999c0e7780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9998009870 tx=0x7f999802b7d0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.119+0000 7fc20cc00640 1 -- 172.21.15.49:0/2425751432 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204015070 con 0x7fc2100b7a80 2024-04-15T15:18:55.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.119+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9da80b78f0 con 0x7f9da80d2c30 2024-04-15T15:18:55.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.119+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9da80d3e40 con 0x7f9da815ec40 2024-04-15T15:18:55.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.119+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9da80b7a70 con 0x7f9da80d1300 2024-04-15T15:18:55.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7fc20cc00640 1 -- 172.21.15.49:0/2425751432 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc204007a30 con 0x7fc2100b7a80 2024-04-15T15:18:55.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7fc20cc00640 1 -- 172.21.15.49:0/2425751432 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204007e20 con 0x7fc2100b7a80 2024-04-15T15:18:55.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7fc217800640 1 -- 172.21.15.49:0/2425751432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21014ea60 con 0x7fc2100b7a80 2024-04-15T15:18:55.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f9da6800640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da815ec40 0x7f9da81615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f9da5400640 1 -- 172.21.15.49:0/1864501056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9da80d1300 msgr2=0x7f9da8153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f9da5e00640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9da80d1300 0x7f9da8153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f9da5400640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9da80d1300 0x7f9da8153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f9da5400640 1 -- 172.21.15.49:0/1864501056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da815ec40 msgr2=0x7f9da81615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f9da5400640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da815ec40 0x7f9da81615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f9da5400640 1 -- 172.21.15.49:0/1864501056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da8156300 con 0x7f9da80d2c30 2024-04-15T15:18:55.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f9da6800640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da815ec40 0x7f9da81615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:55.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f99aa800640 1 -- 172.21.15.49:0/2079287095 shutdown_connections 2024-04-15T15:18:55.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7fc217800640 1 -- 172.21.15.49:0/2425751432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21014f020 con 0x7fc2100b7a80 2024-04-15T15:18:55.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f99aa800640 1 --2- 172.21.15.49:0/2079287095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 0x7f999c054140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f99aa800640 1 --2- 172.21.15.49:0/2079287095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999c0e3160 0x7f999c0e7780 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.120+0000 7f99aa800640 1 -- 172.21.15.49:0/2079287095 >> 172.21.15.49:0/2079287095 conn(0x7f999c058ee0 msgr2=0x7f999c0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.121+0000 7f9da5400640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da80d2c30 0x7f9da80cca50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400c4d0 tx=0x7f9d9400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.121+0000 7f99aa800640 1 -- 172.21.15.49:0/2079287095 shutdown_connections 2024-04-15T15:18:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.122+0000 7fba9ca00640 1 -- 172.21.15.49:0/1113615835 shutdown_connections 2024-04-15T15:18:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.122+0000 7fba9ca00640 1 -- 172.21.15.49:0/1113615835 wait complete. 2024-04-15T15:18:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.122+0000 7fc217800640 1 -- 172.21.15.49:0/2425751432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc210068c80 con 0x7fc2100b7a80 2024-04-15T15:18:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.121+0000 7f99aa800640 1 -- 172.21.15.49:0/2079287095 wait complete. 2024-04-15T15:18:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.125+0000 7fba9ca00640 1 Processor -- start 2024-04-15T15:18:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.124+0000 7fc20cc00640 1 -- 172.21.15.49:0/2425751432 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc204018070 con 0x7fc2100b7a80 2024-04-15T15:18:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.125+0000 7fc20cc00640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc1f006c8f0 0x7fc1f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.125+0000 7fba9ca00640 1 -- start start 2024-04-15T15:18:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.125+0000 7fc20cc00640 1 -- 172.21.15.49:0/2425751432 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc20408bc90 con 0x7fc2100b7a80 2024-04-15T15:18:55.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.125+0000 7f9d9e000640 1 -- 172.21.15.49:0/1864501056 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94017070 con 0x7f9da80d2c30 2024-04-15T15:18:55.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.126+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da81565e0 con 0x7f9da80d2c30 2024-04-15T15:18:55.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.126+0000 7fc214e00640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc1f006c8f0 0x7fc1f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.126+0000 7f9d9e000640 1 -- 172.21.15.49:0/1864501056 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94005ce0 con 0x7f9da80d2c30 2024-04-15T15:18:55.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.126+0000 7f9d9e000640 1 -- 172.21.15.49:0/1864501056 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94017070 con 0x7f9da80d2c30 2024-04-15T15:18:55.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.126+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da8156ba0 con 0x7f9da80d2c30 2024-04-15T15:18:55.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.126+0000 7f99aa800640 1 Processor -- start 2024-04-15T15:18:55.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.127+0000 7f270ea00640 1 -- 172.21.15.49:0/1762315435 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27180ad340 con 0x7f27100ead00 2024-04-15T15:18:55.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.126+0000 7fba9ca00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba980b5530 0x7fba9814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.126+0000 7fba9ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba980b5b00 0x7fba9815e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.126+0000 7fba9ca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba98162dd0 0x7fba9815e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.127+0000 7fba9ca00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fba980cf8f0 con 0x7fba98162dd0 2024-04-15T15:18:55.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.127+0000 7fba9ca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fba980cf770 con 0x7fba980b5b00 2024-04-15T15:18:55.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.127+0000 7fba9ca00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fba980cfa70 con 0x7fba980b5530 2024-04-15T15:18:55.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.128+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da8068c80 con 0x7f9da80d2c30 2024-04-15T15:18:55.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.128+0000 7fba97400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba980b5530 0x7fba9814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.128+0000 7fba97400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba980b5530 0x7fba9814d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:43190/0 (socket says 172.21.15.49:43190) 2024-04-15T15:18:55.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.128+0000 7fba97400640 1 -- 172.21.15.49:0/209386842 learned_addr learned my addr 172.21.15.49:0/209386842 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.128+0000 7fc214e00640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc1f006c8f0 0x7fc1f006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0079a0 tx=0x7fc1fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7fba96a00640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba980b5b00 0x7fba9815e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.127+0000 7f99aa800640 1 -- start start 2024-04-15T15:18:55.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7f99aa800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f999c03c0a0 0x7f999c16cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7f99aa800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 0x7f999c03da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7f99aa800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999c0e3160 0x7f999c03df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7f99aa800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f999c0e3540 con 0x7f999c03c0a0 2024-04-15T15:18:55.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7f99aa800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f999c006640 con 0x7f999c0e3160 2024-04-15T15:18:55.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7f99aa800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f999c006b60 con 0x7f999c053cb0 2024-04-15T15:18:55.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7f99a9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f999c03c0a0 0x7f999c16cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7f99a9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f999c03c0a0 0x7f999c16cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39852/0 (socket says 172.21.15.49:39852) 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7f99a9400640 1 -- 172.21.15.49:0/3908028830 learned_addr learned my addr 172.21.15.49:0/3908028830 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.130+0000 7f99a8a00640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 0x7f999c03da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.130+0000 7f99a9400640 1 -- 172.21.15.49:0/3908028830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 msgr2=0x7f999c03da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.130+0000 7f99a9400640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 0x7f999c03da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.130+0000 7f99a9e00640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999c0e3160 0x7f999c03df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.130+0000 7f99a9400640 1 -- 172.21.15.49:0/3908028830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999c0e3160 msgr2=0x7f999c03df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.130+0000 7f99a9400640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999c0e3160 0x7f999c03df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7fba97400640 1 -- 172.21.15.49:0/209386842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba980b5b00 msgr2=0x7fba9815e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7fba97400640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba980b5b00 0x7fba9815e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7fba97400640 1 -- 172.21.15.49:0/209386842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba98162dd0 msgr2=0x7fba9815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7fba97400640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba98162dd0 0x7fba9815e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.129+0000 7fba97400640 1 -- 172.21.15.49:0/209386842 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba98157d30 con 0x7fba980b5530 2024-04-15T15:18:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.130+0000 7f99a9400640 1 -- 172.21.15.49:0/3908028830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c051ae0 con 0x7f999c03c0a0 2024-04-15T15:18:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.130+0000 7f99a9e00640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999c0e3160 0x7f999c03df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.130+0000 7fba96a00640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba980b5b00 0x7fba9815e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:55.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.131+0000 7f99a9400640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f999c03c0a0 0x7f999c16cc20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9998009870 tx=0x7f99980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.131+0000 7fba97400640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba980b5530 0x7fba9814d830 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fba8800c750 tx=0x7fba8800cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.130+0000 7f9d9e000640 1 -- 172.21.15.49:0/1864501056 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d9401a070 con 0x7f9da80d2c30 2024-04-15T15:18:55.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.131+0000 7fba8f400640 1 -- 172.21.15.49:0/209386842 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88018070 con 0x7fba980b5530 2024-04-15T15:18:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.132+0000 7f9d9e000640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9d8806caf0 0x7f9d8806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.132+0000 7f9d9e000640 1 -- 172.21.15.49:0/1864501056 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9d9408aa10 con 0x7f9da80d2c30 2024-04-15T15:18:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.132+0000 7f9da5e00640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9d8806caf0 0x7f9d8806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.132+0000 7fba8f400640 1 -- 172.21.15.49:0/209386842 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba880072d0 con 0x7fba980b5530 2024-04-15T15:18:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.133+0000 7fba9ca00640 1 -- 172.21.15.49:0/209386842 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba98158220 con 0x7fba980b5530 2024-04-15T15:18:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.133+0000 7fba9ca00640 1 -- 172.21.15.49:0/209386842 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba981587e0 con 0x7fba980b5530 2024-04-15T15:18:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.133+0000 7f9da5e00640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9d8806caf0 0x7f9d8806efb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9d98036000 tx=0x7f9d98008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.133+0000 7fba8f400640 1 -- 172.21.15.49:0/209386842 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880047e0 con 0x7fba980b5530 2024-04-15T15:18:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.134+0000 7fba9ca00640 1 -- 172.21.15.49:0/209386842 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba980755c0 con 0x7fba980b5530 2024-04-15T15:18:55.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.137+0000 7f9d9e000640 1 -- 172.21.15.49:0/1864501056 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d940587e0 con 0x7f9da80d2c30 2024-04-15T15:18:55.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.136+0000 7f99a1600640 1 -- 172.21.15.49:0/3908028830 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998042070 con 0x7f999c03c0a0 2024-04-15T15:18:55.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.136+0000 7f99aa800640 1 -- 172.21.15.49:0/3908028830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c051dc0 con 0x7f999c03c0a0 2024-04-15T15:18:55.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.136+0000 7f99aa800640 1 -- 172.21.15.49:0/3908028830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c052380 con 0x7f999c03c0a0 2024-04-15T15:18:55.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.137+0000 7f99a1600640 1 -- 172.21.15.49:0/3908028830 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9998030710 con 0x7f999c03c0a0 2024-04-15T15:18:55.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.137+0000 7f99a1600640 1 -- 172.21.15.49:0/3908028830 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998042070 con 0x7f999c03c0a0 2024-04-15T15:18:55.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.137+0000 7fba8f400640 1 -- 172.21.15.49:0/209386842 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba88004980 con 0x7fba980b5530 2024-04-15T15:18:55.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.138+0000 7fba8f400640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fba7806c8f0 0x7fba7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.139+0000 7fba8f400640 1 -- 172.21.15.49:0/209386842 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fba8808b400 con 0x7fba980b5530 2024-04-15T15:18:55.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.140+0000 7fc20cc00640 1 -- 172.21.15.49:0/2425751432 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2040599d0 con 0x7fc2100b7a80 2024-04-15T15:18:55.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.140+0000 7f99aa800640 1 -- 172.21.15.49:0/3908028830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c005390 con 0x7f999c03c0a0 2024-04-15T15:18:55.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.139+0000 7fba96a00640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fba7806c8f0 0x7fba7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.140+0000 7fba96a00640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fba7806c8f0 0x7fba7806edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba84034000 tx=0x7fba84008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.142+0000 7f99a1600640 1 -- 172.21.15.49:0/3908028830 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9998045070 con 0x7f999c03c0a0 2024-04-15T15:18:55.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.144+0000 7f99a1600640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f998406caf0 0x7f998406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.144+0000 7f99a1600640 1 -- 172.21.15.49:0/3908028830 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f99980ad520 con 0x7f999c03c0a0 2024-04-15T15:18:55.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.145+0000 7f99a8a00640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f998406caf0 0x7f998406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.146+0000 7f99a8a00640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f998406caf0 0x7f998406efb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f999c00e150 tx=0x7f9990009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.149+0000 7fba8f400640 1 -- 172.21.15.49:0/209386842 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba88004c70 con 0x7fba980b5530 2024-04-15T15:18:55.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.150+0000 7f99a1600640 1 -- 172.21.15.49:0/3908028830 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f999807b8b0 con 0x7f999c03c0a0 2024-04-15T15:18:55.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.150+0000 7f0ea0e00640 1 Processor -- start 2024-04-15T15:18:55.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.150+0000 7f0ea0e00640 1 -- start start 2024-04-15T15:18:55.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.152+0000 7f0ea0e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e9c15eab0 0x7f0e9c162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.152+0000 7f0ea0e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9c0d1c60 0x7f0e9c0d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.152+0000 7f0ea0e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e9c0b78f0 con 0x7f0e9c162b70 2024-04-15T15:18:55.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.152+0000 7f0ea0e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9c0d2e50 con 0x7f0e9c15eab0 2024-04-15T15:18:55.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.152+0000 7f0ea0e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9c163010 con 0x7f0e9c0d1c60 2024-04-15T15:18:55.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.153+0000 7f0e9b400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e9c15eab0 0x7f0e9c162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.154+0000 7f0e9b400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e9c15eab0 0x7f0e9c162630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39868/0 (socket says 172.21.15.49:39868) 2024-04-15T15:18:55.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.154+0000 7f0e9b400640 1 -- 172.21.15.49:0/4108422866 learned_addr learned my addr 172.21.15.49:0/4108422866 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.154+0000 7f0e9b400640 1 -- 172.21.15.49:0/4108422866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9c0d1c60 msgr2=0x7f0e9c0d32e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-15T15:18:55.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.154+0000 7f0e9b400640 1 --2- 172.21.15.49:0/4108422866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9c0d1c60 0x7f0e9c0d32e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.155+0000 7f0e9b400640 1 -- 172.21.15.49:0/4108422866 >> v1:172.21.15.107:6789/0 conn(0x7f0e9c162b70 legacy=0x7f0e9c0d1550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-15T15:18:55.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.155+0000 7f0e9b400640 1 -- 172.21.15.49:0/4108422866 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c1632f0 con 0x7f0e9c15eab0 2024-04-15T15:18:55.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.158+0000 7f0e9b400640 1 --2- 172.21.15.49:0/4108422866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 0x7f0e9c162630 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009e50 tx=0x7f0e9402bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8546394f75cd256 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.162+0000 7f0e9a000640 1 -- 172.21.15.49:0/4108422866 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9402caa0 con 0x7f0e9c15eab0 2024-04-15T15:18:55.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.162+0000 7f0e9a000640 1 -- 172.21.15.49:0/4108422866 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9402cc40 con 0x7f0e9c15eab0 2024-04-15T15:18:55.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.162+0000 7f0e9a000640 1 -- 172.21.15.49:0/4108422866 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e940343f0 con 0x7f0e9c15eab0 2024-04-15T15:18:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.163+0000 7f0ea0e00640 1 -- 172.21.15.49:0/4108422866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 msgr2=0x7f0e9c162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.164+0000 7f0ea0e00640 1 --2- 172.21.15.49:0/4108422866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 0x7f0e9c162630 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009e50 tx=0x7f0e9402bf00 comp rx=0 tx=0).stop 2024-04-15T15:18:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.164+0000 7f0ea0e00640 1 -- 172.21.15.49:0/4108422866 shutdown_connections 2024-04-15T15:18:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.164+0000 7f0ea0e00640 1 --2- 172.21.15.49:0/4108422866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9c0d1c60 0x7f0e9c0d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.164+0000 7f0ea0e00640 1 --2- 172.21.15.49:0/4108422866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 0x7f0e9c162630 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.164+0000 7f0ea0e00640 1 -- 172.21.15.49:0/4108422866 >> 172.21.15.49:0/4108422866 conn(0x7f0e9c073240 msgr2=0x7f0e9c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.165+0000 7f0ea0e00640 1 -- 172.21.15.49:0/4108422866 shutdown_connections 2024-04-15T15:18:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.165+0000 7f0ea0e00640 1 -- 172.21.15.49:0/4108422866 wait complete. 2024-04-15T15:18:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.168+0000 7f0ea0e00640 1 Processor -- start 2024-04-15T15:18:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.168+0000 7f0ea0e00640 1 -- start start 2024-04-15T15:18:55.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.170+0000 7f0ea0e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9c0d1c60 0x7f0e9c14b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.170+0000 7f0ea0e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 0x7f0e9c0d3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.170+0000 7f0ea0e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e9c162b70 0x7f0e9c14e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.170+0000 7f0ea0e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c0ba590 con 0x7f0e9c15eab0 2024-04-15T15:18:55.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.170+0000 7f0ea0e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c15ee90 con 0x7f0e9c162b70 2024-04-15T15:18:55.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.170+0000 7f0ea0e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9c0ba710 con 0x7f0e9c0d1c60 2024-04-15T15:18:55.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.171+0000 7f0e9aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 0x7f0e9c0d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.171+0000 7f0e9b400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9c0d1c60 0x7f0e9c14b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.171+0000 7f0e9aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 0x7f0e9c0d3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39872/0 (socket says 172.21.15.49:39872) 2024-04-15T15:18:55.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.171+0000 7f0e9aa00640 1 -- 172.21.15.49:0/2288601839 learned_addr learned my addr 172.21.15.49:0/2288601839 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.171+0000 7f0e9be00640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e9c162b70 0x7f0e9c14e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.171+0000 7f0e9aa00640 1 -- 172.21.15.49:0/2288601839 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9c0d1c60 msgr2=0x7f0e9c14b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.171+0000 7f0e9aa00640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9c0d1c60 0x7f0e9c14b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.171+0000 7f0e9aa00640 1 -- 172.21.15.49:0/2288601839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e9c162b70 msgr2=0x7f0e9c14e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.171+0000 7f0e9aa00640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e9c162b70 0x7f0e9c14e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.171+0000 7f0e9aa00640 1 -- 172.21.15.49:0/2288601839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c0cd4f0 con 0x7f0e9c15eab0 2024-04-15T15:18:55.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.172+0000 7f0e9aa00640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 0x7f0e9c0d3910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800b410 tx=0x7f0e8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.173+0000 7f0e93400640 1 -- 172.21.15.49:0/2288601839 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88015020 con 0x7f0e9c15eab0 2024-04-15T15:18:55.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.173+0000 7f0ea0e00640 1 -- 172.21.15.49:0/2288601839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c0cd7d0 con 0x7f0e9c15eab0 2024-04-15T15:18:55.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.173+0000 7f0ea0e00640 1 -- 172.21.15.49:0/2288601839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c14bd70 con 0x7f0e9c15eab0 2024-04-15T15:18:55.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.173+0000 7f0e93400640 1 -- 172.21.15.49:0/2288601839 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e88011830 con 0x7f0e9c15eab0 2024-04-15T15:18:55.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.173+0000 7f0e93400640 1 -- 172.21.15.49:0/2288601839 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88015020 con 0x7f0e9c15eab0 2024-04-15T15:18:55.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.177+0000 7f0ea0e00640 1 -- 172.21.15.49:0/2288601839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e74005290 con 0x7f0e9c15eab0 2024-04-15T15:18:55.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.177+0000 7f0e93400640 1 -- 172.21.15.49:0/2288601839 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e88018020 con 0x7f0e9c15eab0 2024-04-15T15:18:55.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.179+0000 7f0e93400640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7c06caf0 0x7f0e7c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.179+0000 7f0e93400640 1 -- 172.21.15.49:0/2288601839 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0e8808b5a0 con 0x7f0e9c15eab0 2024-04-15T15:18:55.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.180+0000 7f0e9b400640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7c06caf0 0x7f0e7c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.183+0000 7f0e9b400640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7c06caf0 0x7f0e7c06efb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e94035000 tx=0x7f0e94030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.189+0000 7f0e93400640 1 -- 172.21.15.49:0/2288601839 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e8801f020 con 0x7f0e9c15eab0 2024-04-15T15:18:55.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.531+0000 7f3d62000640 1 -- 172.21.15.49:0/332411782 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3d6c144de0 con 0x7f3d6c147fe0 2024-04-15T15:18:55.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.532+0000 7f3d68c00640 1 -- 172.21.15.49:0/332411782 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d5c059aa0 con 0x7f3d6c147fe0 2024-04-15T15:18:55.534 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:55.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.535+0000 7f3d62000640 1 -- 172.21.15.49:0/332411782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3d4c06c920 msgr2=0x7f3d4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.535+0000 7f3d62000640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3d4c06c920 0x7f3d4c06ede0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d58036000 tx=0x7f3d5803d040 comp rx=0 tx=0).stop 2024-04-15T15:18:55.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.536+0000 7f3d62000640 1 -- 172.21.15.49:0/332411782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c147fe0 msgr2=0x7f3d6c145000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.536+0000 7f3d62000640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c147fe0 0x7f3d6c145000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00e870 tx=0x7f3d5c00ed30 comp rx=0 tx=0).stop 2024-04-15T15:18:55.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.536+0000 7f3d62000640 1 -- 172.21.15.49:0/332411782 shutdown_connections 2024-04-15T15:18:55.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.536+0000 7f3d62000640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3d4c06c920 0x7f3d4c06ede0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.536+0000 7f3d62000640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d6c147fe0 0x7f3d6c145000 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.536+0000 7f3d62000640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d6c1483c0 0x7f3d6c1448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.536+0000 7f3d62000640 1 --2- 172.21.15.49:0/332411782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6c0aec60 0x7f3d6c146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.536+0000 7f3d62000640 1 -- 172.21.15.49:0/332411782 >> 172.21.15.49:0/332411782 conn(0x7f3d6c067b70 msgr2=0x7f3d6c06f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.537+0000 7f3d62000640 1 -- 172.21.15.49:0/332411782 shutdown_connections 2024-04-15T15:18:55.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.537+0000 7f3d62000640 1 -- 172.21.15.49:0/332411782 wait complete. 2024-04-15T15:18:55.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.550+0000 7f9b36400640 1 -- 172.21.15.49:0/507279726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9b280ede30 con 0x7f9b28053cb0 2024-04-15T15:18:55.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.551+0000 7f9b2cc00640 1 -- 172.21.15.49:0/507279726 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9b1c058dd0 con 0x7f9b28053cb0 2024-04-15T15:18:55.556 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.554+0000 7f9b36400640 1 -- 172.21.15.49:0/507279726 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9b1006c970 msgr2=0x7f9b1006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.554+0000 7f9b36400640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9b1006c970 0x7f9b1006ee30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b24034000 tx=0x7f9b24008030 comp rx=0 tx=0).stop 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.554+0000 7f9b36400640 1 -- 172.21.15.49:0/507279726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b28053cb0 msgr2=0x7f9b280d92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.554+0000 7f9b36400640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b28053cb0 0x7f9b280d92f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00c8a0 tx=0x7f9b1c00cd60 comp rx=0 tx=0).stop 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.555+0000 7f9b36400640 1 -- 172.21.15.49:0/507279726 shutdown_connections 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.555+0000 7f9b36400640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9b1006c970 0x7f9b1006ee30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.555+0000 7f9b36400640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b280e3160 0x7f9b2803d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.555+0000 7f9b36400640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b28053cb0 0x7f9b280d92f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.555+0000 7f9b36400640 1 --2- 172.21.15.49:0/507279726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b2803c0a0 0x7f9b28186b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.555+0000 7f9b36400640 1 -- 172.21.15.49:0/507279726 >> 172.21.15.49:0/507279726 conn(0x7f9b28058ee0 msgr2=0x7f9b28038940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.555+0000 7f9b36400640 1 -- 172.21.15.49:0/507279726 shutdown_connections 2024-04-15T15:18:55.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.555+0000 7f9b36400640 1 -- 172.21.15.49:0/507279726 wait complete. 2024-04-15T15:18:55.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-04-15T15:18:55.567 DEBUG:teuthology.parallel:result is None 2024-04-15T15:18:55.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.575+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9da806bf60 con 0x7f9da80d2c30 2024-04-15T15:18:55.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.576+0000 7f9d9e000640 1 -- 172.21.15.49:0/1864501056 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d9405cae0 con 0x7f9da80d2c30 2024-04-15T15:18:55.579 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:55.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.578+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9d8806caf0 msgr2=0x7f9d8806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.579+0000 7f9dacc00640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9d8806caf0 0x7f9d8806efb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9d98036000 tx=0x7f9d98008030 comp rx=0 tx=0).stop 2024-04-15T15:18:55.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.579+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da80d2c30 msgr2=0x7f9da80cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.579+0000 7f9dacc00640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da80d2c30 0x7f9da80cca50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400c4d0 tx=0x7f9d9400c990 comp rx=0 tx=0).stop 2024-04-15T15:18:55.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.579+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 shutdown_connections 2024-04-15T15:18:55.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.579+0000 7f9dacc00640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9d8806caf0 0x7f9d8806efb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.579+0000 7f9dacc00640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da815ec40 0x7f9da81615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.579+0000 7f9dacc00640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da80d2c30 0x7f9da80cca50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.579+0000 7f9dacc00640 1 --2- 172.21.15.49:0/1864501056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9da80d1300 0x7f9da8153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.579+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 >> 172.21.15.49:0/1864501056 conn(0x7f9da8073690 msgr2=0x7f9da80cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.580+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 shutdown_connections 2024-04-15T15:18:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.580+0000 7f9dacc00640 1 -- 172.21.15.49:0/1864501056 wait complete. 2024-04-15T15:18:55.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-04-15T15:18:55.583 DEBUG:teuthology.parallel:result is None 2024-04-15T15:18:55.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.586+0000 7f2717e00640 1 -- 172.21.15.49:0/1762315435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f271004cdb0 con 0x7f27100ead00 2024-04-15T15:18:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.586+0000 7f270ea00640 1 -- 172.21.15.49:0/1762315435 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f27180f6060 con 0x7f27100ead00 2024-04-15T15:18:55.588 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2024-04-15T15:18:55.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.589+0000 7fc217800640 1 -- 172.21.15.49:0/2425751432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc2100b3bf0 con 0x7fc2100b7a80 2024-04-15T15:18:55.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.590+0000 7f2717e00640 1 -- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f26f806caf0 msgr2=0x7f26f806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.590+0000 7fc20cc00640 1 -- 172.21.15.49:0/2425751432 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc20405dcd0 con 0x7fc2100b7a80 2024-04-15T15:18:55.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.590+0000 7f2717e00640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f26f806caf0 0x7f26f806efb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2708001a40 tx=0x7f2708001710 comp rx=0 tx=0).stop 2024-04-15T15:18:55.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.590+0000 7f2717e00640 1 -- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27100ead00 msgr2=0x7f2710048960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.590+0000 7f2717e00640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27100ead00 0x7f2710048960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2718065750 tx=0x7f271806b5b0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.593 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:55.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.591+0000 7f2717e00640 1 -- 172.21.15.49:0/1762315435 shutdown_connections 2024-04-15T15:18:55.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.591+0000 7f2717e00640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f26f806caf0 0x7f26f806efb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.591+0000 7f2717e00640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27100ead00 0x7f2710048960 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.591+0000 7f2717e00640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27100069e0 0x7f2710048420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.591+0000 7f2717e00640 1 --2- 172.21.15.49:0/1762315435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2710005e90 0x7f27100d3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.591+0000 7f2717e00640 1 -- 172.21.15.49:0/1762315435 >> 172.21.15.49:0/1762315435 conn(0x7f2710009fb0 msgr2=0x7f2710033bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.591+0000 7f2717e00640 1 -- 172.21.15.49:0/1762315435 shutdown_connections 2024-04-15T15:18:55.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.591+0000 7f2717e00640 1 -- 172.21.15.49:0/1762315435 wait complete. 2024-04-15T15:18:55.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.591+0000 7f99aa800640 1 -- 172.21.15.49:0/3908028830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f999c056bf0 con 0x7f999c03c0a0 2024-04-15T15:18:55.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.592+0000 7f99a1600640 1 -- 172.21.15.49:0/3908028830 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f999807b1b0 con 0x7f999c03c0a0 2024-04-15T15:18:55.595 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:55.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.594+0000 7fc217800640 1 -- 172.21.15.49:0/2425751432 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc1f006c8f0 msgr2=0x7fc1f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.594+0000 7fc217800640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc1f006c8f0 0x7fc1f006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0079a0 tx=0x7fc1fc008040 comp rx=0 tx=0).stop 2024-04-15T15:18:55.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.594+0000 7fc217800640 1 -- 172.21.15.49:0/2425751432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2100b7a80 msgr2=0x7fc21014f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.594+0000 7fc217800640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2100b7a80 0x7fc21014f820 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc20400c820 tx=0x7fc20400cce0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.594+0000 7fc217800640 1 -- 172.21.15.49:0/2425751432 shutdown_connections 2024-04-15T15:18:55.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.594+0000 7fc217800640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc1f006c8f0 0x7fc1f006edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.594+0000 7fc217800640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc21015ec60 0x7fc210151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.594+0000 7fc217800640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2100b7a80 0x7fc21014f820 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.594+0000 7fc217800640 1 --2- 172.21.15.49:0/2425751432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2100b63d0 0x7fc21014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.594+0000 7fc217800640 1 -- 172.21.15.49:0/2425751432 >> 172.21.15.49:0/2425751432 conn(0x7fc210073480 msgr2=0x7fc2100cae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.595+0000 7fc217800640 1 -- 172.21.15.49:0/2425751432 shutdown_connections 2024-04-15T15:18:55.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.595+0000 7fc217800640 1 -- 172.21.15.49:0/2425751432 wait complete. 2024-04-15T15:18:55.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.595+0000 7f99aa800640 1 -- 172.21.15.49:0/3908028830 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f998406caf0 msgr2=0x7f998406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.595+0000 7f99aa800640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f998406caf0 0x7f998406efb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f999c00e150 tx=0x7f9990009040 comp rx=0 tx=0).stop 2024-04-15T15:18:55.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.595+0000 7f99aa800640 1 -- 172.21.15.49:0/3908028830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f999c03c0a0 msgr2=0x7f999c16cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.595+0000 7f99aa800640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f999c03c0a0 0x7f999c16cc20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9998009870 tx=0x7f99980046c0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.601+0000 7fba9ca00640 1 -- 172.21.15.49:0/209386842 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fba981664f0 con 0x7fba980b5530 2024-04-15T15:18:55.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.601+0000 7f99aa800640 1 -- 172.21.15.49:0/3908028830 shutdown_connections 2024-04-15T15:18:55.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.601+0000 7f99aa800640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f998406caf0 0x7f998406efb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.601+0000 7f99aa800640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f999c053cb0 0x7f999c03da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.601+0000 7f99aa800640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999c0e3160 0x7f999c03df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.601+0000 7f99aa800640 1 --2- 172.21.15.49:0/3908028830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f999c03c0a0 0x7f999c16cc20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.601+0000 7f99aa800640 1 -- 172.21.15.49:0/3908028830 >> 172.21.15.49:0/3908028830 conn(0x7f999c058ee0 msgr2=0x7f999c0e0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.601+0000 7fba8f400640 1 -- 172.21.15.49:0/209386842 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fba88059790 con 0x7fba980b5530 2024-04-15T15:18:55.604 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:55.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.603+0000 7f99aa800640 1 -- 172.21.15.49:0/3908028830 shutdown_connections 2024-04-15T15:18:55.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.603+0000 7f99aa800640 1 -- 172.21.15.49:0/3908028830 wait complete. 2024-04-15T15:18:55.606 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-04-15T15:18:55.606 DEBUG:teuthology.parallel:result is None 2024-04-15T15:18:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 -- 172.21.15.49:0/209386842 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fba7806c8f0 msgr2=0x7fba7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fba7806c8f0 0x7fba7806edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba84034000 tx=0x7fba84008030 comp rx=0 tx=0).stop 2024-04-15T15:18:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 -- 172.21.15.49:0/209386842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba980b5530 msgr2=0x7fba9814d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba980b5530 0x7fba9814d830 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fba8800c750 tx=0x7fba8800cc10 comp rx=0 tx=0).stop 2024-04-15T15:18:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 -- 172.21.15.49:0/209386842 shutdown_connections 2024-04-15T15:18:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fba7806c8f0 0x7fba7806edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba98162dd0 0x7fba9815e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba980b5b00 0x7fba9815e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 --2- 172.21.15.49:0/209386842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba980b5530 0x7fba9814d830 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 -- 172.21.15.49:0/209386842 >> 172.21.15.49:0/209386842 conn(0x7fba980b23a0 msgr2=0x7fba981615d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 -- 172.21.15.49:0/209386842 shutdown_connections 2024-04-15T15:18:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.604+0000 7fba9ca00640 1 -- 172.21.15.49:0/209386842 wait complete. 2024-04-15T15:18:55.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2024-04-15T15:18:55.615 DEBUG:teuthology.parallel:result is None 2024-04-15T15:18:55.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-04-15T15:18:55.623 DEBUG:teuthology.parallel:result is None 2024-04-15T15:18:55.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2024-04-15T15:18:55.624 DEBUG:teuthology.parallel:result is None 2024-04-15T15:18:55.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2024-04-15T15:18:55.625 DEBUG:teuthology.parallel:result is None 2024-04-15T15:18:55.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.627+0000 7f0ea0e00640 1 -- 172.21.15.49:0/2288601839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0e74005580 con 0x7f0e9c15eab0 2024-04-15T15:18:55.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.627+0000 7f0e93400640 1 -- 172.21.15.49:0/2288601839 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e880592e0 con 0x7f0e9c15eab0 2024-04-15T15:18:55.629 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2024-04-15T15:18:55.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.629+0000 7f0ea0e00640 1 -- 172.21.15.49:0/2288601839 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7c06caf0 msgr2=0x7f0e7c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.629+0000 7f0ea0e00640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7c06caf0 0x7f0e7c06efb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e94035000 tx=0x7f0e94030000 comp rx=0 tx=0).stop 2024-04-15T15:18:55.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.629+0000 7f0ea0e00640 1 -- 172.21.15.49:0/2288601839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 msgr2=0x7f0e9c0d3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.629+0000 7f0ea0e00640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 0x7f0e9c0d3910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800b410 tx=0x7f0e8800b8d0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.629+0000 7f0ea0e00640 1 -- 172.21.15.49:0/2288601839 shutdown_connections 2024-04-15T15:18:55.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.629+0000 7f0ea0e00640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7c06caf0 0x7f0e7c06efb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.629+0000 7f0ea0e00640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e9c162b70 0x7f0e9c14e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.629+0000 7f0ea0e00640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e9c15eab0 0x7f0e9c0d3910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.629+0000 7f0ea0e00640 1 --2- 172.21.15.49:0/2288601839 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9c0d1c60 0x7f0e9c14b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.629+0000 7f0ea0e00640 1 -- 172.21.15.49:0/2288601839 >> 172.21.15.49:0/2288601839 conn(0x7f0e9c073240 msgr2=0x7f0e9c0c8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.630+0000 7f0ea0e00640 1 -- 172.21.15.49:0/2288601839 shutdown_connections 2024-04-15T15:18:55.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.630+0000 7f0ea0e00640 1 -- 172.21.15.49:0/2288601839 wait complete. 2024-04-15T15:18:55.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2024-04-15T15:18:55.649 DEBUG:teuthology.parallel:result is None 2024-04-15T15:18:55.649 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-15T15:18:55.649 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:18:55.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.909+0000 7fb91d200640 1 Processor -- start 2024-04-15T15:18:55.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.910+0000 7fb91d200640 1 -- start start 2024-04-15T15:18:55.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb91d200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb91815ec50 0x7fb91806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb91d200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb9180b3ab0 0x7fb91806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb91d200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb918067720 con 0x7fb91806be70 2024-04-15T15:18:55.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb91d200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb91806c2c0 con 0x7fb9180b3ab0 2024-04-15T15:18:55.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb91d200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fb91815f030 con 0x7fb91815ec50 2024-04-15T15:18:55.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb90fe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb9180b3ab0 0x7fb91806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb90fe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb9180b3ab0 0x7fb91806b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39886/0 (socket says 172.21.15.49:39886) 2024-04-15T15:18:55.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb90fe00640 1 -- 172.21.15.49:0/2998671560 learned_addr learned my addr 172.21.15.49:0/2998671560 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb90fe00640 1 -- 172.21.15.49:0/2998671560 >> v2:172.21.15.107:3300/0 conn(0x7fb91815ec50 msgr2=0x7fb91806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:55.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb90fe00640 1 --2- 172.21.15.49:0/2998671560 >> v2:172.21.15.107:3300/0 conn(0x7fb91815ec50 0x7fb91806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb90fe00640 1 -- 172.21.15.49:0/2998671560 >> v1:172.21.15.107:6789/0 conn(0x7fb91806be70 legacy=0x7fb91806f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:55.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.912+0000 7fb90fe00640 1 -- 172.21.15.49:0/2998671560 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9180679c0 con 0x7fb9180b3ab0 2024-04-15T15:18:55.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.914+0000 7fb90fe00640 1 --2- 172.21.15.49:0/2998671560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9180b3ab0 0x7fb91806b930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb91806dfe0 tx=0x7fb90002b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f52bab443a28e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.914+0000 7fb915400640 1 -- 172.21.15.49:0/2998671560 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900038070 con 0x7fb9180b3ab0 2024-04-15T15:18:55.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.914+0000 7fb915400640 1 -- 172.21.15.49:0/2998671560 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb900033620 con 0x7fb9180b3ab0 2024-04-15T15:18:55.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.914+0000 7fb915400640 1 -- 172.21.15.49:0/2998671560 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9000339f0 con 0x7fb9180b3ab0 2024-04-15T15:18:55.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.914+0000 7fb91d200640 1 -- 172.21.15.49:0/2998671560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9180b3ab0 msgr2=0x7fb91806b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.914+0000 7fb91d200640 1 --2- 172.21.15.49:0/2998671560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9180b3ab0 0x7fb91806b930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb91806dfe0 tx=0x7fb90002b7d0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.915+0000 7fb91d200640 1 -- 172.21.15.49:0/2998671560 shutdown_connections 2024-04-15T15:18:55.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.915+0000 7fb91d200640 1 --2- 172.21.15.49:0/2998671560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9180b3ab0 0x7fb91806b930 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.915+0000 7fb91d200640 1 --2- 172.21.15.49:0/2998671560 >> v2:172.21.15.107:3300/0 conn(0x7fb91815ec50 0x7fb91806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.915+0000 7fb91d200640 1 -- 172.21.15.49:0/2998671560 >> 172.21.15.49:0/2998671560 conn(0x7fb9180b6700 msgr2=0x7fb9180b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:55.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.915+0000 7fb91d200640 1 -- 172.21.15.49:0/2998671560 shutdown_connections 2024-04-15T15:18:55.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.915+0000 7fb91d200640 1 -- 172.21.15.49:0/2998671560 wait complete. 2024-04-15T15:18:55.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.916+0000 7fb91d200640 1 Processor -- start 2024-04-15T15:18:55.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.917+0000 7fb91d200640 1 -- start start 2024-04-15T15:18:55.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.917+0000 7fb91d200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb91806be70 0x7fb91814db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.917+0000 7fb91d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9180b3ab0 0x7fb91814d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb91d200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb91815ec50 0x7fb9181e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb91d200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb91806eff0 con 0x7fb91806be70 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb91d200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb91806ed10 con 0x7fb9180b3ab0 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb91d200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb91806d1b0 con 0x7fb91815ec50 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb915e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb91806be70 0x7fb91814db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb915e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb91806be70 0x7fb91814db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39894/0 (socket says 172.21.15.49:39894) 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb915e00640 1 -- 172.21.15.49:0/4212601979 learned_addr learned my addr 172.21.15.49:0/4212601979 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb90fe00640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9180b3ab0 0x7fb91814d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb916800640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb91815ec50 0x7fb9181e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb915e00640 1 -- 172.21.15.49:0/4212601979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb91815ec50 msgr2=0x7fb9181e73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb915e00640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb91815ec50 0x7fb9181e73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb915e00640 1 -- 172.21.15.49:0/4212601979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9180b3ab0 msgr2=0x7fb91814d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb915e00640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9180b3ab0 0x7fb91814d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:55.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.918+0000 7fb915e00640 1 -- 172.21.15.49:0/4212601979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9180b37c0 con 0x7fb91806be70 2024-04-15T15:18:55.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.919+0000 7fb915e00640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb91806be70 0x7fb91814db60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b4a0 tx=0x7fb90800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.919+0000 7fb90e000640 1 -- 172.21.15.49:0/4212601979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908014070 con 0x7fb91806be70 2024-04-15T15:18:55.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.919+0000 7fb90e000640 1 -- 172.21.15.49:0/4212601979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb908010610 con 0x7fb91806be70 2024-04-15T15:18:55.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.919+0000 7fb91d200640 1 -- 172.21.15.49:0/4212601979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9180b2690 con 0x7fb91806be70 2024-04-15T15:18:55.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.919+0000 7fb90e000640 1 -- 172.21.15.49:0/4212601979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908010a00 con 0x7fb91806be70 2024-04-15T15:18:55.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.919+0000 7fb91d200640 1 -- 172.21.15.49:0/4212601979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb918157d50 con 0x7fb91806be70 2024-04-15T15:18:55.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.920+0000 7fb91d200640 1 -- 172.21.15.49:0/4212601979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb918068c80 con 0x7fb91806be70 2024-04-15T15:18:55.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.921+0000 7fb90e000640 1 -- 172.21.15.49:0/4212601979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb908017070 con 0x7fb91806be70 2024-04-15T15:18:55.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.922+0000 7fb90e000640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb8f406c920 0x7fb8f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:55.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.922+0000 7fb90e000640 1 -- 172.21.15.49:0/4212601979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb90808b640 con 0x7fb91806be70 2024-04-15T15:18:55.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.922+0000 7fb90fe00640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb8f406c920 0x7fb8f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:55.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.923+0000 7fb90fe00640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb8f406c920 0x7fb8f406ede0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb900035000 tx=0x7fb900008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:55.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:55.927+0000 7fb90e000640 1 -- 172.21.15.49:0/4212601979 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9080599d0 con 0x7fb91806be70 2024-04-15T15:18:56.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.122+0000 7fb91d200640 1 -- 172.21.15.49:0/4212601979 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb91814d990 con 0x7fb8f406c920 2024-04-15T15:18:56.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.140+0000 7fb90e000640 1 -- 172.21.15.49:0/4212601979 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251839 (secure 0 0 0) 0x7fb91814d990 con 0x7fb8f406c920 2024-04-15T15:18:56.144 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:56.147 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:18:56.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.147+0000 7fb91d200640 1 -- 172.21.15.49:0/4212601979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb8f406c920 msgr2=0x7fb8f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.147+0000 7fb91d200640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb8f406c920 0x7fb8f406ede0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb900035000 tx=0x7fb900008030 comp rx=0 tx=0).stop 2024-04-15T15:18:56.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.147+0000 7fb91d200640 1 -- 172.21.15.49:0/4212601979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb91806be70 msgr2=0x7fb91814db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.147+0000 7fb91d200640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb91806be70 0x7fb91814db60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b4a0 tx=0x7fb90800b960 comp rx=0 tx=0).stop 2024-04-15T15:18:56.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.147+0000 7fb91d200640 1 -- 172.21.15.49:0/4212601979 shutdown_connections 2024-04-15T15:18:56.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.147+0000 7fb91d200640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb8f406c920 0x7fb8f406ede0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.147+0000 7fb91d200640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb91815ec50 0x7fb9181e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.147+0000 7fb91d200640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9180b3ab0 0x7fb91814d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.147+0000 7fb91d200640 1 --2- 172.21.15.49:0/4212601979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb91806be70 0x7fb91814db60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.147+0000 7fb91d200640 1 -- 172.21.15.49:0/4212601979 >> 172.21.15.49:0/4212601979 conn(0x7fb9180b6700 msgr2=0x7fb918159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.148+0000 7fb91d200640 1 -- 172.21.15.49:0/4212601979 shutdown_connections 2024-04-15T15:18:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.148+0000 7fb91d200640 1 -- 172.21.15.49:0/4212601979 wait complete. 2024-04-15T15:18:56.167 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-15T15:18:54.773844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235952,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738768,"statfs":{"total":773094113280,"available":772852498432,"internally_reserved":0,"allocated":3055616,"data_stored":1816820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":62,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.964382"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.825719+0000","last_change":"2024-04-15T15:18:45.850244+0000","last_active":"2024-04-15T15:18:46.825719+0000","last_peered":"2024-04-15T15:18:46.825719+0000","last_clean":"2024-04-15T15:18:46.825719+0000","last_became_active":"2024-04-15T15:18:45.849857+0000","last_became_peered":"2024-04-15T15:18:45.849857+0000","last_unstale":"2024-04-15T15:18:46.825719+0000","last_undegraded":"2024-04-15T15:18:46.825719+0000","last_fullsized":"2024-04-15T15:18:46.825719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:08:04.293347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870331+0000","last_change":"2024-04-15T15:18:45.843425+0000","last_active":"2024-04-15T15:18:46.870331+0000","last_peered":"2024-04-15T15:18:46.870331+0000","last_clean":"2024-04-15T15:18:46.870331+0000","last_became_active":"2024-04-15T15:18:45.843052+0000","last_became_peered":"2024-04-15T15:18:45.843052+0000","last_unstale":"2024-04-15T15:18:46.870331+0000","last_undegraded":"2024-04-15T15:18:46.870331+0000","last_fullsized":"2024-04-15T15:18:46.870331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868186+0000","last_change":"2024-04-15T15:18:45.853920+0000","last_active":"2024-04-15T15:18:47.868186+0000","last_peered":"2024-04-15T15:18:47.868186+0000","last_clean":"2024-04-15T15:18:47.868186+0000","last_became_active":"2024-04-15T15:18:45.853659+0000","last_became_peered":"2024-04-15T15:18:45.853659+0000","last_unstale":"2024-04-15T15:18:47.868186+0000","last_undegraded":"2024-04-15T15:18:47.868186+0000","last_fullsized":"2024-04-15T15:18:47.868186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823670+0000","last_change":"2024-04-15T15:18:45.861695+0000","last_active":"2024-04-15T15:18:46.823670+0000","last_peered":"2024-04-15T15:18:46.823670+0000","last_clean":"2024-04-15T15:18:46.823670+0000","last_became_active":"2024-04-15T15:18:45.858560+0000","last_became_peered":"2024-04-15T15:18:45.858560+0000","last_unstale":"2024-04-15T15:18:46.823670+0000","last_undegraded":"2024-04-15T15:18:46.823670+0000","last_fullsized":"2024-04-15T15:18:46.823670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:29:14.132066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820061+0000","last_change":"2024-04-15T15:18:45.849530+0000","last_active":"2024-04-15T15:18:46.820061+0000","last_peered":"2024-04-15T15:18:46.820061+0000","last_clean":"2024-04-15T15:18:46.820061+0000","last_became_active":"2024-04-15T15:18:45.849255+0000","last_became_peered":"2024-04-15T15:18:45.849255+0000","last_unstale":"2024-04-15T15:18:46.820061+0000","last_undegraded":"2024-04-15T15:18:46.820061+0000","last_fullsized":"2024-04-15T15:18:46.820061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:36:03.614062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822193+0000","last_change":"2024-04-15T15:18:45.842283+0000","last_active":"2024-04-15T15:18:46.822193+0000","last_peered":"2024-04-15T15:18:46.822193+0000","last_clean":"2024-04-15T15:18:46.822193+0000","last_became_active":"2024-04-15T15:18:45.842004+0000","last_became_peered":"2024-04-15T15:18:45.842004+0000","last_unstale":"2024-04-15T15:18:46.822193+0000","last_undegraded":"2024-04-15T15:18:46.822193+0000","last_fullsized":"2024-04-15T15:18:46.822193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:08:29.977812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818174+0000","last_change":"2024-04-15T15:18:45.856124+0000","last_active":"2024-04-15T15:18:46.818174+0000","last_peered":"2024-04-15T15:18:46.818174+0000","last_clean":"2024-04-15T15:18:46.818174+0000","last_became_active":"2024-04-15T15:18:45.849034+0000","last_became_peered":"2024-04-15T15:18:45.849034+0000","last_unstale":"2024-04-15T15:18:46.818174+0000","last_undegraded":"2024-04-15T15:18:46.818174+0000","last_fullsized":"2024-04-15T15:18:46.818174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:49:43.342170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.866494+0000","last_change":"2024-04-15T15:18:45.848306+0000","last_active":"2024-04-15T15:18:47.866494+0000","last_peered":"2024-04-15T15:18:47.866494+0000","last_clean":"2024-04-15T15:18:47.866494+0000","last_became_active":"2024-04-15T15:18:45.848033+0000","last_became_peered":"2024-04-15T15:18:45.848033+0000","last_unstale":"2024-04-15T15:18:47.866494+0000","last_undegraded":"2024-04-15T15:18:47.866494+0000","last_fullsized":"2024-04-15T15:18:47.866494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.826189+0000","last_change":"2024-04-15T15:18:45.862452+0000","last_active":"2024-04-15T15:18:46.826189+0000","last_peered":"2024-04-15T15:18:46.826189+0000","last_clean":"2024-04-15T15:18:46.826189+0000","last_became_active":"2024-04-15T15:18:45.862157+0000","last_became_peered":"2024-04-15T15:18:45.862157+0000","last_unstale":"2024-04-15T15:18:46.826189+0000","last_undegraded":"2024-04-15T15:18:46.826189+0000","last_fullsized":"2024-04-15T15:18:46.826189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:14:17.068828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875290+0000","last_change":"2024-04-15T15:18:45.855709+0000","last_active":"2024-04-15T15:18:46.875290+0000","last_peered":"2024-04-15T15:18:46.875290+0000","last_clean":"2024-04-15T15:18:46.875290+0000","last_became_active":"2024-04-15T15:18:45.855243+0000","last_became_peered":"2024-04-15T15:18:45.855243+0000","last_unstale":"2024-04-15T15:18:46.875290+0000","last_undegraded":"2024-04-15T15:18:46.875290+0000","last_fullsized":"2024-04-15T15:18:46.875290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874937+0000","last_change":"2024-04-15T15:18:45.859684+0000","last_active":"2024-04-15T15:18:46.874937+0000","last_peered":"2024-04-15T15:18:46.874937+0000","last_clean":"2024-04-15T15:18:46.874937+0000","last_became_active":"2024-04-15T15:18:45.856683+0000","last_became_peered":"2024-04-15T15:18:45.856683+0000","last_unstale":"2024-04-15T15:18:46.874937+0000","last_undegraded":"2024-04-15T15:18:46.874937+0000","last_fullsized":"2024-04-15T15:18:46.874937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822415+0000","last_change":"2024-04-15T15:18:45.849769+0000","last_active":"2024-04-15T15:18:46.822415+0000","last_peered":"2024-04-15T15:18:46.822415+0000","last_clean":"2024-04-15T15:18:46.822415+0000","last_became_active":"2024-04-15T15:18:45.849462+0000","last_became_peered":"2024-04-15T15:18:45.849462+0000","last_unstale":"2024-04-15T15:18:46.822415+0000","last_undegraded":"2024-04-15T15:18:46.822415+0000","last_fullsized":"2024-04-15T15:18:46.822415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:35:53.211120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868757+0000","last_change":"2024-04-15T15:18:45.852551+0000","last_active":"2024-04-15T15:18:47.868757+0000","last_peered":"2024-04-15T15:18:47.868757+0000","last_clean":"2024-04-15T15:18:47.868757+0000","last_became_active":"2024-04-15T15:18:45.851658+0000","last_became_peered":"2024-04-15T15:18:45.851658+0000","last_unstale":"2024-04-15T15:18:47.868757+0000","last_undegraded":"2024-04-15T15:18:47.868757+0000","last_fullsized":"2024-04-15T15:18:47.868757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:49:43.342170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.866096+0000","last_change":"2024-04-15T15:18:45.842127+0000","last_active":"2024-04-15T15:18:47.866096+0000","last_peered":"2024-04-15T15:18:47.866096+0000","last_clean":"2024-04-15T15:18:47.866096+0000","last_became_active":"2024-04-15T15:18:45.841662+0000","last_became_peered":"2024-04-15T15:18:45.841662+0000","last_unstale":"2024-04-15T15:18:47.866096+0000","last_undegraded":"2024-04-15T15:18:47.866096+0000","last_fullsized":"2024-04-15T15:18:47.866096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:39:00.386699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869891+0000","last_change":"2024-04-15T15:18:45.832097+0000","last_active":"2024-04-15T15:18:46.869891+0000","last_peered":"2024-04-15T15:18:46.869891+0000","last_clean":"2024-04-15T15:18:46.869891+0000","last_became_active":"2024-04-15T15:18:45.831411+0000","last_became_peered":"2024-04-15T15:18:45.831411+0000","last_unstale":"2024-04-15T15:18:46.869891+0000","last_undegraded":"2024-04-15T15:18:46.869891+0000","last_fullsized":"2024-04-15T15:18:46.869891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822037+0000","last_change":"2024-04-15T15:18:45.846022+0000","last_active":"2024-04-15T15:18:46.822037+0000","last_peered":"2024-04-15T15:18:46.822037+0000","last_clean":"2024-04-15T15:18:46.822037+0000","last_became_active":"2024-04-15T15:18:45.844025+0000","last_became_peered":"2024-04-15T15:18:45.844025+0000","last_unstale":"2024-04-15T15:18:46.822037+0000","last_undegraded":"2024-04-15T15:18:46.822037+0000","last_fullsized":"2024-04-15T15:18:46.822037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.825219+0000","last_change":"2024-04-15T15:18:45.849397+0000","last_active":"2024-04-15T15:18:46.825219+0000","last_peered":"2024-04-15T15:18:46.825219+0000","last_clean":"2024-04-15T15:18:46.825219+0000","last_became_active":"2024-04-15T15:18:45.849114+0000","last_became_peered":"2024-04-15T15:18:45.849114+0000","last_unstale":"2024-04-15T15:18:46.825219+0000","last_undegraded":"2024-04-15T15:18:46.825219+0000","last_fullsized":"2024-04-15T15:18:46.825219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:08:04.293347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.864639+0000","last_change":"2024-04-15T15:18:45.848118+0000","last_active":"2024-04-15T15:18:47.864639+0000","last_peered":"2024-04-15T15:18:47.864639+0000","last_clean":"2024-04-15T15:18:47.864639+0000","last_became_active":"2024-04-15T15:18:45.847644+0000","last_became_peered":"2024-04-15T15:18:45.847644+0000","last_unstale":"2024-04-15T15:18:47.864639+0000","last_undegraded":"2024-04-15T15:18:47.864639+0000","last_fullsized":"2024-04-15T15:18:47.864639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:22:54.875123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820072+0000","last_change":"2024-04-15T15:18:45.860524+0000","last_active":"2024-04-15T15:18:46.820072+0000","last_peered":"2024-04-15T15:18:46.820072+0000","last_clean":"2024-04-15T15:18:46.820072+0000","last_became_active":"2024-04-15T15:18:45.852020+0000","last_became_peered":"2024-04-15T15:18:45.852020+0000","last_unstale":"2024-04-15T15:18:46.820072+0000","last_undegraded":"2024-04-15T15:18:46.820072+0000","last_fullsized":"2024-04-15T15:18:46.820072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:06:25.954239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874881+0000","last_change":"2024-04-15T15:18:45.852135+0000","last_active":"2024-04-15T15:18:46.874881+0000","last_peered":"2024-04-15T15:18:46.874881+0000","last_clean":"2024-04-15T15:18:46.874881+0000","last_became_active":"2024-04-15T15:18:45.851841+0000","last_became_peered":"2024-04-15T15:18:45.851841+0000","last_unstale":"2024-04-15T15:18:46.874881+0000","last_undegraded":"2024-04-15T15:18:46.874881+0000","last_fullsized":"2024-04-15T15:18:46.874881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868076+0000","last_change":"2024-04-15T15:18:45.859189+0000","last_active":"2024-04-15T15:18:47.868076+0000","last_peered":"2024-04-15T15:18:47.868076+0000","last_clean":"2024-04-15T15:18:47.868076+0000","last_became_active":"2024-04-15T15:18:45.857117+0000","last_became_peered":"2024-04-15T15:18:45.857117+0000","last_unstale":"2024-04-15T15:18:47.868076+0000","last_undegraded":"2024-04-15T15:18:47.868076+0000","last_fullsized":"2024-04-15T15:18:47.868076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822607+0000","last_change":"2024-04-15T15:18:45.840913+0000","last_active":"2024-04-15T15:18:46.822607+0000","last_peered":"2024-04-15T15:18:46.822607+0000","last_clean":"2024-04-15T15:18:46.822607+0000","last_became_active":"2024-04-15T15:18:45.840257+0000","last_became_peered":"2024-04-15T15:18:45.840257+0000","last_unstale":"2024-04-15T15:18:46.822607+0000","last_undegraded":"2024-04-15T15:18:46.822607+0000","last_fullsized":"2024-04-15T15:18:46.822607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:49:43.342170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.867493+0000","last_change":"2024-04-15T15:18:45.851064+0000","last_active":"2024-04-15T15:18:47.867493+0000","last_peered":"2024-04-15T15:18:47.867493+0000","last_clean":"2024-04-15T15:18:47.867493+0000","last_became_active":"2024-04-15T15:18:45.850725+0000","last_became_peered":"2024-04-15T15:18:45.850725+0000","last_unstale":"2024-04-15T15:18:47.867493+0000","last_undegraded":"2024-04-15T15:18:47.867493+0000","last_fullsized":"2024-04-15T15:18:47.867493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820869+0000","last_change":"2024-04-15T15:18:45.838476+0000","last_active":"2024-04-15T15:18:46.820869+0000","last_peered":"2024-04-15T15:18:46.820869+0000","last_clean":"2024-04-15T15:18:46.820869+0000","last_became_active":"2024-04-15T15:18:45.838160+0000","last_became_peered":"2024-04-15T15:18:45.838160+0000","last_unstale":"2024-04-15T15:18:46.820869+0000","last_undegraded":"2024-04-15T15:18:46.820869+0000","last_fullsized":"2024-04-15T15:18:46.820869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868577+0000","last_change":"2024-04-15T15:18:45.855537+0000","last_active":"2024-04-15T15:18:47.868577+0000","last_peered":"2024-04-15T15:18:47.868577+0000","last_clean":"2024-04-15T15:18:47.868577+0000","last_became_active":"2024-04-15T15:18:45.855144+0000","last_became_peered":"2024-04-15T15:18:45.855144+0000","last_unstale":"2024-04-15T15:18:47.868577+0000","last_undegraded":"2024-04-15T15:18:47.868577+0000","last_fullsized":"2024-04-15T15:18:47.868577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875245+0000","last_change":"2024-04-15T15:18:45.848420+0000","last_active":"2024-04-15T15:18:46.875245+0000","last_peered":"2024-04-15T15:18:46.875245+0000","last_clean":"2024-04-15T15:18:46.875245+0000","last_became_active":"2024-04-15T15:18:45.848080+0000","last_became_peered":"2024-04-15T15:18:45.848080+0000","last_unstale":"2024-04-15T15:18:46.875245+0000","last_undegraded":"2024-04-15T15:18:46.875245+0000","last_fullsized":"2024-04-15T15:18:46.875245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869996+0000","last_change":"2024-04-15T15:18:45.858358+0000","last_active":"2024-04-15T15:18:46.869996+0000","last_peered":"2024-04-15T15:18:46.869996+0000","last_clean":"2024-04-15T15:18:46.869996+0000","last_became_active":"2024-04-15T15:18:45.856289+0000","last_became_peered":"2024-04-15T15:18:45.856289+0000","last_unstale":"2024-04-15T15:18:46.869996+0000","last_undegraded":"2024-04-15T15:18:46.869996+0000","last_fullsized":"2024-04-15T15:18:46.869996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868753+0000","last_change":"2024-04-15T15:18:45.854349+0000","last_active":"2024-04-15T15:18:47.868753+0000","last_peered":"2024-04-15T15:18:47.868753+0000","last_clean":"2024-04-15T15:18:47.868753+0000","last_became_active":"2024-04-15T15:18:45.853978+0000","last_became_peered":"2024-04-15T15:18:45.853978+0000","last_unstale":"2024-04-15T15:18:47.868753+0000","last_undegraded":"2024-04-15T15:18:47.868753+0000","last_fullsized":"2024-04-15T15:18:47.868753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:08:29.977812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.867237+0000","last_change":"2024-04-15T15:18:45.858661+0000","last_active":"2024-04-15T15:18:47.867237+0000","last_peered":"2024-04-15T15:18:47.867237+0000","last_clean":"2024-04-15T15:18:47.867237+0000","last_became_active":"2024-04-15T15:18:45.858244+0000","last_became_peered":"2024-04-15T15:18:45.858244+0000","last_unstale":"2024-04-15T15:18:47.867237+0000","last_undegraded":"2024-04-15T15:18:47.867237+0000","last_fullsized":"2024-04-15T15:18:47.867237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823029+0000","last_change":"2024-04-15T15:18:45.840590+0000","last_active":"2024-04-15T15:18:46.823029+0000","last_peered":"2024-04-15T15:18:46.823029+0000","last_clean":"2024-04-15T15:18:46.823029+0000","last_became_active":"2024-04-15T15:18:45.839849+0000","last_became_peered":"2024-04-15T15:18:45.839849+0000","last_unstale":"2024-04-15T15:18:46.823029+0000","last_undegraded":"2024-04-15T15:18:46.823029+0000","last_fullsized":"2024-04-15T15:18:46.823029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.566254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.864522+0000","last_change":"2024-04-15T15:18:45.847940+0000","last_active":"2024-04-15T15:18:47.864522+0000","last_peered":"2024-04-15T15:18:47.864522+0000","last_clean":"2024-04-15T15:18:47.864522+0000","last_became_active":"2024-04-15T15:18:45.846916+0000","last_became_peered":"2024-04-15T15:18:45.846916+0000","last_unstale":"2024-04-15T15:18:47.864522+0000","last_undegraded":"2024-04-15T15:18:47.864522+0000","last_fullsized":"2024-04-15T15:18:47.864522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:27.690982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874483+0000","last_change":"2024-04-15T15:18:45.846327+0000","last_active":"2024-04-15T15:18:46.874483+0000","last_peered":"2024-04-15T15:18:46.874483+0000","last_clean":"2024-04-15T15:18:46.874483+0000","last_became_active":"2024-04-15T15:18:45.846010+0000","last_became_peered":"2024-04-15T15:18:45.846010+0000","last_unstale":"2024-04-15T15:18:46.874483+0000","last_undegraded":"2024-04-15T15:18:46.874483+0000","last_fullsized":"2024-04-15T15:18:46.874483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820924+0000","last_change":"2024-04-15T15:18:45.846764+0000","last_active":"2024-04-15T15:18:46.820924+0000","last_peered":"2024-04-15T15:18:46.820924+0000","last_clean":"2024-04-15T15:18:46.820924+0000","last_became_active":"2024-04-15T15:18:45.842870+0000","last_became_peered":"2024-04-15T15:18:45.842870+0000","last_unstale":"2024-04-15T15:18:46.820924+0000","last_undegraded":"2024-04-15T15:18:46.820924+0000","last_fullsized":"2024-04-15T15:18:46.820924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:38:23.700029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.869023+0000","last_change":"2024-04-15T15:18:44.864457+0000","last_active":"2024-04-15T15:18:47.869023+0000","last_peered":"2024-04-15T15:18:47.869023+0000","last_clean":"2024-04-15T15:18:47.869023+0000","last_became_active":"2024-04-15T15:18:44.863999+0000","last_became_peered":"2024-04-15T15:18:44.863999+0000","last_unstale":"2024-04-15T15:18:47.869023+0000","last_undegraded":"2024-04-15T15:18:47.869023+0000","last_fullsized":"2024-04-15T15:18:47.869023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.867390+0000","last_change":"2024-04-15T15:18:45.855613+0000","last_active":"2024-04-15T15:18:47.867390+0000","last_peered":"2024-04-15T15:18:47.867390+0000","last_clean":"2024-04-15T15:18:47.867390+0000","last_became_active":"2024-04-15T15:18:45.855327+0000","last_became_peered":"2024-04-15T15:18:45.855327+0000","last_unstale":"2024-04-15T15:18:47.867390+0000","last_undegraded":"2024-04-15T15:18:47.867390+0000","last_fullsized":"2024-04-15T15:18:47.867390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874502+0000","last_change":"2024-04-15T15:18:44.864183+0000","last_active":"2024-04-15T15:18:46.874502+0000","last_peered":"2024-04-15T15:18:46.874502+0000","last_clean":"2024-04-15T15:18:46.874502+0000","last_became_active":"2024-04-15T15:18:44.861740+0000","last_became_peered":"2024-04-15T15:18:44.861740+0000","last_unstale":"2024-04-15T15:18:46.874502+0000","last_undegraded":"2024-04-15T15:18:46.874502+0000","last_fullsized":"2024-04-15T15:18:46.874502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.867367+0000","last_change":"2024-04-15T15:18:44.867636+0000","last_active":"2024-04-15T15:18:47.867367+0000","last_peered":"2024-04-15T15:18:47.867367+0000","last_clean":"2024-04-15T15:18:47.867367+0000","last_became_active":"2024-04-15T15:18:44.867307+0000","last_became_peered":"2024-04-15T15:18:44.867307+0000","last_unstale":"2024-04-15T15:18:47.867367+0000","last_undegraded":"2024-04-15T15:18:47.867367+0000","last_fullsized":"2024-04-15T15:18:47.867367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.865159+0000","last_change":"2024-04-15T15:18:45.850873+0000","last_active":"2024-04-15T15:18:47.865159+0000","last_peered":"2024-04-15T15:18:47.865159+0000","last_clean":"2024-04-15T15:18:47.865159+0000","last_became_active":"2024-04-15T15:18:45.850532+0000","last_became_peered":"2024-04-15T15:18:45.850532+0000","last_unstale":"2024-04-15T15:18:47.865159+0000","last_undegraded":"2024-04-15T15:18:47.865159+0000","last_fullsized":"2024-04-15T15:18:47.865159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821134+0000","last_change":"2024-04-15T15:18:44.866286+0000","last_active":"2024-04-15T15:18:46.821134+0000","last_peered":"2024-04-15T15:18:46.821134+0000","last_clean":"2024-04-15T15:18:46.821134+0000","last_became_active":"2024-04-15T15:18:44.863387+0000","last_became_peered":"2024-04-15T15:18:44.863387+0000","last_unstale":"2024-04-15T15:18:46.821134+0000","last_undegraded":"2024-04-15T15:18:46.821134+0000","last_fullsized":"2024-04-15T15:18:46.821134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869851+0000","last_change":"2024-04-15T15:18:45.839256+0000","last_active":"2024-04-15T15:18:46.869851+0000","last_peered":"2024-04-15T15:18:46.869851+0000","last_clean":"2024-04-15T15:18:46.869851+0000","last_became_active":"2024-04-15T15:18:45.838960+0000","last_became_peered":"2024-04-15T15:18:45.838960+0000","last_unstale":"2024-04-15T15:18:46.869851+0000","last_undegraded":"2024-04-15T15:18:46.869851+0000","last_fullsized":"2024-04-15T15:18:46.869851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875122+0000","last_change":"2024-04-15T15:18:44.870228+0000","last_active":"2024-04-15T15:18:46.875122+0000","last_peered":"2024-04-15T15:18:46.875122+0000","last_clean":"2024-04-15T15:18:46.875122+0000","last_became_active":"2024-04-15T15:18:44.869155+0000","last_became_peered":"2024-04-15T15:18:44.869155+0000","last_unstale":"2024-04-15T15:18:46.875122+0000","last_undegraded":"2024-04-15T15:18:46.875122+0000","last_fullsized":"2024-04-15T15:18:46.875122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824756+0000","last_change":"2024-04-15T15:18:45.845321+0000","last_active":"2024-04-15T15:18:46.824756+0000","last_peered":"2024-04-15T15:18:46.824756+0000","last_clean":"2024-04-15T15:18:46.824756+0000","last_became_active":"2024-04-15T15:18:45.840422+0000","last_became_peered":"2024-04-15T15:18:45.840422+0000","last_unstale":"2024-04-15T15:18:46.824756+0000","last_undegraded":"2024-04-15T15:18:46.824756+0000","last_fullsized":"2024-04-15T15:18:46.824756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:32:58.852361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824852+0000","last_change":"2024-04-15T15:18:44.865670+0000","last_active":"2024-04-15T15:18:46.824852+0000","last_peered":"2024-04-15T15:18:46.824852+0000","last_clean":"2024-04-15T15:18:46.824852+0000","last_became_active":"2024-04-15T15:18:44.864795+0000","last_became_peered":"2024-04-15T15:18:44.864795+0000","last_unstale":"2024-04-15T15:18:46.824852+0000","last_undegraded":"2024-04-15T15:18:46.824852+0000","last_fullsized":"2024-04-15T15:18:46.824852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.869467+0000","last_change":"2024-04-15T15:18:45.852169+0000","last_active":"2024-04-15T15:18:47.869467+0000","last_peered":"2024-04-15T15:18:47.869467+0000","last_clean":"2024-04-15T15:18:47.869467+0000","last_became_active":"2024-04-15T15:18:45.845297+0000","last_became_peered":"2024-04-15T15:18:45.845297+0000","last_unstale":"2024-04-15T15:18:47.869467+0000","last_undegraded":"2024-04-15T15:18:47.869467+0000","last_fullsized":"2024-04-15T15:18:47.869467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:08:29.977812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870182+0000","last_change":"2024-04-15T15:18:44.866158+0000","last_active":"2024-04-15T15:18:46.870182+0000","last_peered":"2024-04-15T15:18:46.870182+0000","last_clean":"2024-04-15T15:18:46.870182+0000","last_became_active":"2024-04-15T15:18:44.865766+0000","last_became_peered":"2024-04-15T15:18:44.865766+0000","last_unstale":"2024-04-15T15:18:46.870182+0000","last_undegraded":"2024-04-15T15:18:46.870182+0000","last_fullsized":"2024-04-15T15:18:46.870182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822687+0000","last_change":"2024-04-15T15:18:45.843966+0000","last_active":"2024-04-15T15:18:46.822687+0000","last_peered":"2024-04-15T15:18:46.822687+0000","last_clean":"2024-04-15T15:18:46.822687+0000","last_became_active":"2024-04-15T15:18:45.842648+0000","last_became_peered":"2024-04-15T15:18:45.842648+0000","last_unstale":"2024-04-15T15:18:46.822687+0000","last_undegraded":"2024-04-15T15:18:46.822687+0000","last_fullsized":"2024-04-15T15:18:46.822687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:39:50.034981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874917+0000","last_change":"2024-04-15T15:18:44.867158+0000","last_active":"2024-04-15T15:18:46.874917+0000","last_peered":"2024-04-15T15:18:46.874917+0000","last_clean":"2024-04-15T15:18:46.874917+0000","last_became_active":"2024-04-15T15:18:44.866840+0000","last_became_peered":"2024-04-15T15:18:44.866840+0000","last_unstale":"2024-04-15T15:18:46.874917+0000","last_undegraded":"2024-04-15T15:18:46.874917+0000","last_fullsized":"2024-04-15T15:18:46.874917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818675+0000","last_change":"2024-04-15T15:18:45.860752+0000","last_active":"2024-04-15T15:18:46.818675+0000","last_peered":"2024-04-15T15:18:46.818675+0000","last_clean":"2024-04-15T15:18:46.818675+0000","last_became_active":"2024-04-15T15:18:45.860381+0000","last_became_peered":"2024-04-15T15:18:45.860381+0000","last_unstale":"2024-04-15T15:18:46.818675+0000","last_undegraded":"2024-04-15T15:18:46.818675+0000","last_fullsized":"2024-04-15T15:18:46.818675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:53.984837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.867491+0000","last_change":"2024-04-15T15:18:44.865018+0000","last_active":"2024-04-15T15:18:47.867491+0000","last_peered":"2024-04-15T15:18:47.867491+0000","last_clean":"2024-04-15T15:18:47.867491+0000","last_became_active":"2024-04-15T15:18:44.864073+0000","last_became_peered":"2024-04-15T15:18:44.864073+0000","last_unstale":"2024-04-15T15:18:47.867491+0000","last_undegraded":"2024-04-15T15:18:47.867491+0000","last_fullsized":"2024-04-15T15:18:47.867491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:34.796362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.866527+0000","last_change":"2024-04-15T15:18:45.860189+0000","last_active":"2024-04-15T15:18:47.866527+0000","last_peered":"2024-04-15T15:18:47.866527+0000","last_clean":"2024-04-15T15:18:47.866527+0000","last_became_active":"2024-04-15T15:18:45.859786+0000","last_became_peered":"2024-04-15T15:18:45.859786+0000","last_unstale":"2024-04-15T15:18:47.866527+0000","last_undegraded":"2024-04-15T15:18:47.866527+0000","last_fullsized":"2024-04-15T15:18:47.866527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822942+0000","last_change":"2024-04-15T15:18:44.866125+0000","last_active":"2024-04-15T15:18:46.822942+0000","last_peered":"2024-04-15T15:18:46.822942+0000","last_clean":"2024-04-15T15:18:46.822942+0000","last_became_active":"2024-04-15T15:18:44.864849+0000","last_became_peered":"2024-04-15T15:18:44.864849+0000","last_unstale":"2024-04-15T15:18:46.822942+0000","last_undegraded":"2024-04-15T15:18:46.822942+0000","last_fullsized":"2024-04-15T15:18:46.822942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819575+0000","last_change":"2024-04-15T15:18:45.852862+0000","last_active":"2024-04-15T15:18:46.819575+0000","last_peered":"2024-04-15T15:18:46.819575+0000","last_clean":"2024-04-15T15:18:46.819575+0000","last_became_active":"2024-04-15T15:18:45.851610+0000","last_became_peered":"2024-04-15T15:18:45.851610+0000","last_unstale":"2024-04-15T15:18:46.819575+0000","last_undegraded":"2024-04-15T15:18:46.819575+0000","last_fullsized":"2024-04-15T15:18:46.819575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:36.969295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874519+0000","last_change":"2024-04-15T15:18:44.871366+0000","last_active":"2024-04-15T15:18:46.874519+0000","last_peered":"2024-04-15T15:18:46.874519+0000","last_clean":"2024-04-15T15:18:46.874519+0000","last_became_active":"2024-04-15T15:18:44.870865+0000","last_became_peered":"2024-04-15T15:18:44.870865+0000","last_unstale":"2024-04-15T15:18:46.874519+0000","last_undegraded":"2024-04-15T15:18:46.874519+0000","last_fullsized":"2024-04-15T15:18:46.874519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874458+0000","last_change":"2024-04-15T15:18:45.847642+0000","last_active":"2024-04-15T15:18:46.874458+0000","last_peered":"2024-04-15T15:18:46.874458+0000","last_clean":"2024-04-15T15:18:46.874458+0000","last_became_active":"2024-04-15T15:18:45.845790+0000","last_became_peered":"2024-04-15T15:18:45.845790+0000","last_unstale":"2024-04-15T15:18:46.874458+0000","last_undegraded":"2024-04-15T15:18:46.874458+0000","last_fullsized":"2024-04-15T15:18:46.874458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.863926+0000","last_change":"2024-04-15T15:18:44.857029+0000","last_active":"2024-04-15T15:18:47.863926+0000","last_peered":"2024-04-15T15:18:47.863926+0000","last_clean":"2024-04-15T15:18:47.863926+0000","last_became_active":"2024-04-15T15:18:44.855203+0000","last_became_peered":"2024-04-15T15:18:44.855203+0000","last_unstale":"2024-04-15T15:18:47.863926+0000","last_undegraded":"2024-04-15T15:18:47.863926+0000","last_fullsized":"2024-04-15T15:18:47.863926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:34.796362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818209+0000","last_change":"2024-04-15T15:18:45.856661+0000","last_active":"2024-04-15T15:18:46.818209+0000","last_peered":"2024-04-15T15:18:46.818209+0000","last_clean":"2024-04-15T15:18:46.818209+0000","last_became_active":"2024-04-15T15:18:45.856443+0000","last_became_peered":"2024-04-15T15:18:45.856443+0000","last_unstale":"2024-04-15T15:18:46.818209+0000","last_undegraded":"2024-04-15T15:18:46.818209+0000","last_fullsized":"2024-04-15T15:18:46.818209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.566254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822757+0000","last_change":"2024-04-15T15:18:44.866401+0000","last_active":"2024-04-15T15:18:46.822757+0000","last_peered":"2024-04-15T15:18:46.822757+0000","last_clean":"2024-04-15T15:18:46.822757+0000","last_became_active":"2024-04-15T15:18:44.866099+0000","last_became_peered":"2024-04-15T15:18:44.866099+0000","last_unstale":"2024-04-15T15:18:46.822757+0000","last_undegraded":"2024-04-15T15:18:46.822757+0000","last_fullsized":"2024-04-15T15:18:46.822757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:15:03.621969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820130+0000","last_change":"2024-04-15T15:18:45.854183+0000","last_active":"2024-04-15T15:18:46.820130+0000","last_peered":"2024-04-15T15:18:46.820130+0000","last_clean":"2024-04-15T15:18:46.820130+0000","last_became_active":"2024-04-15T15:18:45.853850+0000","last_became_peered":"2024-04-15T15:18:45.853850+0000","last_unstale":"2024-04-15T15:18:46.820130+0000","last_undegraded":"2024-04-15T15:18:46.820130+0000","last_fullsized":"2024-04-15T15:18:46.820130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:32:58.852361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.864715+0000","last_change":"2024-04-15T15:18:44.857499+0000","last_active":"2024-04-15T15:18:47.864715+0000","last_peered":"2024-04-15T15:18:47.864715+0000","last_clean":"2024-04-15T15:18:47.864715+0000","last_became_active":"2024-04-15T15:18:44.857140+0000","last_became_peered":"2024-04-15T15:18:44.857140+0000","last_unstale":"2024-04-15T15:18:47.864715+0000","last_undegraded":"2024-04-15T15:18:47.864715+0000","last_fullsized":"2024-04-15T15:18:47.864715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.826627+0000","last_change":"2024-04-15T15:18:45.860607+0000","last_active":"2024-04-15T15:18:46.826627+0000","last_peered":"2024-04-15T15:18:46.826627+0000","last_clean":"2024-04-15T15:18:46.826627+0000","last_became_active":"2024-04-15T15:18:45.858730+0000","last_became_peered":"2024-04-15T15:18:45.858730+0000","last_unstale":"2024-04-15T15:18:46.826627+0000","last_undegraded":"2024-04-15T15:18:46.826627+0000","last_fullsized":"2024-04-15T15:18:46.826627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:10:16.717594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870214+0000","last_change":"2024-04-15T15:18:44.858372+0000","last_active":"2024-04-15T15:18:46.870214+0000","last_peered":"2024-04-15T15:18:46.870214+0000","last_clean":"2024-04-15T15:18:46.870214+0000","last_became_active":"2024-04-15T15:18:44.858083+0000","last_became_peered":"2024-04-15T15:18:44.858083+0000","last_unstale":"2024-04-15T15:18:46.870214+0000","last_undegraded":"2024-04-15T15:18:46.870214+0000","last_fullsized":"2024-04-15T15:18:46.870214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824118+0000","last_change":"2024-04-15T15:18:45.864746+0000","last_active":"2024-04-15T15:18:46.824118+0000","last_peered":"2024-04-15T15:18:46.824118+0000","last_clean":"2024-04-15T15:18:46.824118+0000","last_became_active":"2024-04-15T15:18:45.858273+0000","last_became_peered":"2024-04-15T15:18:45.858273+0000","last_unstale":"2024-04-15T15:18:46.824118+0000","last_undegraded":"2024-04-15T15:18:46.824118+0000","last_fullsized":"2024-04-15T15:18:46.824118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:36.969295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.864820+0000","last_change":"2024-04-15T15:18:44.864232+0000","last_active":"2024-04-15T15:18:47.864820+0000","last_peered":"2024-04-15T15:18:47.864820+0000","last_clean":"2024-04-15T15:18:47.864820+0000","last_became_active":"2024-04-15T15:18:44.863837+0000","last_became_peered":"2024-04-15T15:18:44.863837+0000","last_unstale":"2024-04-15T15:18:47.864820+0000","last_undegraded":"2024-04-15T15:18:47.864820+0000","last_fullsized":"2024-04-15T15:18:47.864820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818747+0000","last_change":"2024-04-15T15:18:45.860687+0000","last_active":"2024-04-15T15:18:46.818747+0000","last_peered":"2024-04-15T15:18:46.818747+0000","last_clean":"2024-04-15T15:18:46.818747+0000","last_became_active":"2024-04-15T15:18:45.860227+0000","last_became_peered":"2024-04-15T15:18:45.860227+0000","last_unstale":"2024-04-15T15:18:46.818747+0000","last_undegraded":"2024-04-15T15:18:46.818747+0000","last_fullsized":"2024-04-15T15:18:46.818747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:29:14.132066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.817987+0000","last_change":"2024-04-15T15:18:41.162887+0000","last_active":"2024-04-15T15:18:46.817987+0000","last_peered":"2024-04-15T15:18:46.817987+0000","last_clean":"2024-04-15T15:18:46.817987+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:18:46.817987+0000","last_undegraded":"2024-04-15T15:18:46.817987+0000","last_fullsized":"2024-04-15T15:18:46.817987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_clean_scrub_stamp":"2024-04-15T15:18:39.644354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:32:00.047414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821321+0000","last_change":"2024-04-15T15:18:44.858206+0000","last_active":"2024-04-15T15:18:46.821321+0000","last_peered":"2024-04-15T15:18:46.821321+0000","last_clean":"2024-04-15T15:18:46.821321+0000","last_became_active":"2024-04-15T15:18:44.857694+0000","last_became_peered":"2024-04-15T15:18:44.857694+0000","last_unstale":"2024-04-15T15:18:46.821321+0000","last_undegraded":"2024-04-15T15:18:46.821321+0000","last_fullsized":"2024-04-15T15:18:46.821321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868291+0000","last_change":"2024-04-15T15:18:45.849224+0000","last_active":"2024-04-15T15:18:47.868291+0000","last_peered":"2024-04-15T15:18:47.868291+0000","last_clean":"2024-04-15T15:18:47.868291+0000","last_became_active":"2024-04-15T15:18:45.848961+0000","last_became_peered":"2024-04-15T15:18:45.848961+0000","last_unstale":"2024-04-15T15:18:47.868291+0000","last_undegraded":"2024-04-15T15:18:47.868291+0000","last_fullsized":"2024-04-15T15:18:47.868291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.817850+0000","last_change":"2024-04-15T15:18:44.859202+0000","last_active":"2024-04-15T15:18:46.817850+0000","last_peered":"2024-04-15T15:18:46.817850+0000","last_clean":"2024-04-15T15:18:46.817850+0000","last_became_active":"2024-04-15T15:18:44.858791+0000","last_became_peered":"2024-04-15T15:18:44.858791+0000","last_unstale":"2024-04-15T15:18:46.817850+0000","last_undegraded":"2024-04-15T15:18:46.817850+0000","last_fullsized":"2024-04-15T15:18:46.817850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.865928+0000","last_change":"2024-04-15T15:18:45.851981+0000","last_active":"2024-04-15T15:18:47.865928+0000","last_peered":"2024-04-15T15:18:47.865928+0000","last_clean":"2024-04-15T15:18:47.865928+0000","last_became_active":"2024-04-15T15:18:45.851653+0000","last_became_peered":"2024-04-15T15:18:45.851653+0000","last_unstale":"2024-04-15T15:18:47.865928+0000","last_undegraded":"2024-04-15T15:18:47.865928+0000","last_fullsized":"2024-04-15T15:18:47.865928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:16:52.507783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.864638+0000","last_change":"2024-04-15T15:18:44.849893+0000","last_active":"2024-04-15T15:18:47.864638+0000","last_peered":"2024-04-15T15:18:47.864638+0000","last_clean":"2024-04-15T15:18:47.864638+0000","last_became_active":"2024-04-15T15:18:44.848047+0000","last_became_peered":"2024-04-15T15:18:44.848047+0000","last_unstale":"2024-04-15T15:18:47.864638+0000","last_undegraded":"2024-04-15T15:18:47.864638+0000","last_fullsized":"2024-04-15T15:18:47.864638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869862+0000","last_change":"2024-04-15T15:18:45.844515+0000","last_active":"2024-04-15T15:18:46.869862+0000","last_peered":"2024-04-15T15:18:46.869862+0000","last_clean":"2024-04-15T15:18:46.869862+0000","last_became_active":"2024-04-15T15:18:45.844154+0000","last_became_peered":"2024-04-15T15:18:45.844154+0000","last_unstale":"2024-04-15T15:18:46.869862+0000","last_undegraded":"2024-04-15T15:18:46.869862+0000","last_fullsized":"2024-04-15T15:18:46.869862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823178+0000","last_change":"2024-04-15T15:18:44.862254+0000","last_active":"2024-04-15T15:18:46.823178+0000","last_peered":"2024-04-15T15:18:46.823178+0000","last_clean":"2024-04-15T15:18:46.823178+0000","last_became_active":"2024-04-15T15:18:44.861231+0000","last_became_peered":"2024-04-15T15:18:44.861231+0000","last_unstale":"2024-04-15T15:18:46.823178+0000","last_undegraded":"2024-04-15T15:18:46.823178+0000","last_fullsized":"2024-04-15T15:18:46.823178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821451+0000","last_change":"2024-04-15T15:18:45.847966+0000","last_active":"2024-04-15T15:18:46.821451+0000","last_peered":"2024-04-15T15:18:46.821451+0000","last_clean":"2024-04-15T15:18:46.821451+0000","last_became_active":"2024-04-15T15:18:45.847628+0000","last_became_peered":"2024-04-15T15:18:45.847628+0000","last_unstale":"2024-04-15T15:18:46.821451+0000","last_undegraded":"2024-04-15T15:18:46.821451+0000","last_fullsized":"2024-04-15T15:18:46.821451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:35:55.040215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870108+0000","last_change":"2024-04-15T15:18:44.866681+0000","last_active":"2024-04-15T15:18:46.870108+0000","last_peered":"2024-04-15T15:18:46.870108+0000","last_clean":"2024-04-15T15:18:46.870108+0000","last_became_active":"2024-04-15T15:18:44.866366+0000","last_became_peered":"2024-04-15T15:18:44.866366+0000","last_unstale":"2024-04-15T15:18:46.870108+0000","last_undegraded":"2024-04-15T15:18:46.870108+0000","last_fullsized":"2024-04-15T15:18:46.870108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824652+0000","last_change":"2024-04-15T15:18:45.858129+0000","last_active":"2024-04-15T15:18:46.824652+0000","last_peered":"2024-04-15T15:18:46.824652+0000","last_clean":"2024-04-15T15:18:46.824652+0000","last_became_active":"2024-04-15T15:18:45.857824+0000","last_became_peered":"2024-04-15T15:18:45.857824+0000","last_unstale":"2024-04-15T15:18:46.824652+0000","last_undegraded":"2024-04-15T15:18:46.824652+0000","last_fullsized":"2024-04-15T15:18:46.824652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:36:03.614062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820284+0000","last_change":"2024-04-15T15:18:44.866754+0000","last_active":"2024-04-15T15:18:46.820284+0000","last_peered":"2024-04-15T15:18:46.820284+0000","last_clean":"2024-04-15T15:18:46.820284+0000","last_became_active":"2024-04-15T15:18:44.866479+0000","last_became_peered":"2024-04-15T15:18:44.866479+0000","last_unstale":"2024-04-15T15:18:46.820284+0000","last_undegraded":"2024-04-15T15:18:46.820284+0000","last_fullsized":"2024-04-15T15:18:46.820284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:17:09.815187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870141+0000","last_change":"2024-04-15T15:18:45.844997+0000","last_active":"2024-04-15T15:18:46.870141+0000","last_peered":"2024-04-15T15:18:46.870141+0000","last_clean":"2024-04-15T15:18:46.870141+0000","last_became_active":"2024-04-15T15:18:45.844589+0000","last_became_peered":"2024-04-15T15:18:45.844589+0000","last_unstale":"2024-04-15T15:18:46.870141+0000","last_undegraded":"2024-04-15T15:18:46.870141+0000","last_fullsized":"2024-04-15T15:18:46.870141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.873141+0000","last_change":"2024-04-15T15:18:44.861663+0000","last_active":"2024-04-15T15:18:46.873141+0000","last_peered":"2024-04-15T15:18:46.873141+0000","last_clean":"2024-04-15T15:18:46.873141+0000","last_became_active":"2024-04-15T15:18:44.861347+0000","last_became_peered":"2024-04-15T15:18:44.861347+0000","last_unstale":"2024-04-15T15:18:46.873141+0000","last_undegraded":"2024-04-15T15:18:46.873141+0000","last_fullsized":"2024-04-15T15:18:46.873141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:54.111084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868925+0000","last_change":"2024-04-15T15:18:45.854665+0000","last_active":"2024-04-15T15:18:47.868925+0000","last_peered":"2024-04-15T15:18:47.868925+0000","last_clean":"2024-04-15T15:18:47.868925+0000","last_became_active":"2024-04-15T15:18:45.853958+0000","last_became_peered":"2024-04-15T15:18:45.853958+0000","last_unstale":"2024-04-15T15:18:47.868925+0000","last_undegraded":"2024-04-15T15:18:47.868925+0000","last_fullsized":"2024-04-15T15:18:47.868925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.566254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875345+0000","last_change":"2024-04-15T15:18:44.866078+0000","last_active":"2024-04-15T15:18:46.875345+0000","last_peered":"2024-04-15T15:18:46.875345+0000","last_clean":"2024-04-15T15:18:46.875345+0000","last_became_active":"2024-04-15T15:18:44.865730+0000","last_became_peered":"2024-04-15T15:18:44.865730+0000","last_unstale":"2024-04-15T15:18:46.875345+0000","last_undegraded":"2024-04-15T15:18:46.875345+0000","last_fullsized":"2024-04-15T15:18:46.875345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.869123+0000","last_change":"2024-04-15T15:18:45.841162+0000","last_active":"2024-04-15T15:18:47.869123+0000","last_peered":"2024-04-15T15:18:47.869123+0000","last_clean":"2024-04-15T15:18:47.869123+0000","last_became_active":"2024-04-15T15:18:45.840894+0000","last_became_peered":"2024-04-15T15:18:45.840894+0000","last_unstale":"2024-04-15T15:18:47.869123+0000","last_undegraded":"2024-04-15T15:18:47.869123+0000","last_fullsized":"2024-04-15T15:18:47.869123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819201+0000","last_change":"2024-04-15T15:18:44.862312+0000","last_active":"2024-04-15T15:18:46.819201+0000","last_peered":"2024-04-15T15:18:46.819201+0000","last_clean":"2024-04-15T15:18:46.819201+0000","last_became_active":"2024-04-15T15:18:44.862026+0000","last_became_peered":"2024-04-15T15:18:44.862026+0000","last_unstale":"2024-04-15T15:18:46.819201+0000","last_undegraded":"2024-04-15T15:18:46.819201+0000","last_fullsized":"2024-04-15T15:18:46.819201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822643+0000","last_change":"2024-04-15T15:18:45.847316+0000","last_active":"2024-04-15T15:18:46.822643+0000","last_peered":"2024-04-15T15:18:46.822643+0000","last_clean":"2024-04-15T15:18:46.822643+0000","last_became_active":"2024-04-15T15:18:45.846987+0000","last_became_peered":"2024-04-15T15:18:45.846987+0000","last_unstale":"2024-04-15T15:18:46.822643+0000","last_undegraded":"2024-04-15T15:18:46.822643+0000","last_fullsized":"2024-04-15T15:18:46.822643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:58:24.586889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869976+0000","last_change":"2024-04-15T15:18:44.859986+0000","last_active":"2024-04-15T15:18:46.869976+0000","last_peered":"2024-04-15T15:18:46.869976+0000","last_clean":"2024-04-15T15:18:46.869976+0000","last_became_active":"2024-04-15T15:18:44.859733+0000","last_became_peered":"2024-04-15T15:18:44.859733+0000","last_unstale":"2024-04-15T15:18:46.869976+0000","last_undegraded":"2024-04-15T15:18:46.869976+0000","last_fullsized":"2024-04-15T15:18:46.869976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:26.555787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.865523+0000","last_change":"2024-04-15T15:18:45.846102+0000","last_active":"2024-04-15T15:18:47.865523+0000","last_peered":"2024-04-15T15:18:47.865523+0000","last_clean":"2024-04-15T15:18:47.865523+0000","last_became_active":"2024-04-15T15:18:45.845715+0000","last_became_peered":"2024-04-15T15:18:45.845715+0000","last_unstale":"2024-04-15T15:18:47.865523+0000","last_undegraded":"2024-04-15T15:18:47.865523+0000","last_fullsized":"2024-04-15T15:18:47.865523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869992+0000","last_change":"2024-04-15T15:18:44.862018+0000","last_active":"2024-04-15T15:18:46.869992+0000","last_peered":"2024-04-15T15:18:46.869992+0000","last_clean":"2024-04-15T15:18:46.869992+0000","last_became_active":"2024-04-15T15:18:44.861663+0000","last_became_peered":"2024-04-15T15:18:44.861663+0000","last_unstale":"2024-04-15T15:18:46.869992+0000","last_undegraded":"2024-04-15T15:18:46.869992+0000","last_fullsized":"2024-04-15T15:18:46.869992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868976+0000","last_change":"2024-04-15T15:18:45.852079+0000","last_active":"2024-04-15T15:18:47.868976+0000","last_peered":"2024-04-15T15:18:47.868976+0000","last_clean":"2024-04-15T15:18:47.868976+0000","last_became_active":"2024-04-15T15:18:45.851281+0000","last_became_peered":"2024-04-15T15:18:45.851281+0000","last_unstale":"2024-04-15T15:18:47.868976+0000","last_undegraded":"2024-04-15T15:18:47.868976+0000","last_fullsized":"2024-04-15T15:18:47.868976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:53.984837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819899+0000","last_change":"2024-04-15T15:18:44.867447+0000","last_active":"2024-04-15T15:18:46.819899+0000","last_peered":"2024-04-15T15:18:46.819899+0000","last_clean":"2024-04-15T15:18:46.819899+0000","last_became_active":"2024-04-15T15:18:44.867171+0000","last_became_peered":"2024-04-15T15:18:44.867171+0000","last_unstale":"2024-04-15T15:18:46.819899+0000","last_undegraded":"2024-04-15T15:18:46.819899+0000","last_fullsized":"2024-04-15T15:18:46.819899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:26.555787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874593+0000","last_change":"2024-04-15T15:18:45.861329+0000","last_active":"2024-04-15T15:18:46.874593+0000","last_peered":"2024-04-15T15:18:46.874593+0000","last_clean":"2024-04-15T15:18:46.874593+0000","last_became_active":"2024-04-15T15:18:45.859564+0000","last_became_peered":"2024-04-15T15:18:45.859564+0000","last_unstale":"2024-04-15T15:18:46.874593+0000","last_undegraded":"2024-04-15T15:18:46.874593+0000","last_fullsized":"2024-04-15T15:18:46.874593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824729+0000","last_change":"2024-04-15T15:18:45.831940+0000","last_active":"2024-04-15T15:18:46.824729+0000","last_peered":"2024-04-15T15:18:46.824729+0000","last_clean":"2024-04-15T15:18:46.824729+0000","last_became_active":"2024-04-15T15:18:45.829915+0000","last_became_peered":"2024-04-15T15:18:45.829915+0000","last_unstale":"2024-04-15T15:18:46.824729+0000","last_undegraded":"2024-04-15T15:18:46.824729+0000","last_fullsized":"2024-04-15T15:18:46.824729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:06:25.954239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818726+0000","last_change":"2024-04-15T15:18:44.869358+0000","last_active":"2024-04-15T15:18:46.818726+0000","last_peered":"2024-04-15T15:18:46.818726+0000","last_clean":"2024-04-15T15:18:46.818726+0000","last_became_active":"2024-04-15T15:18:44.869040+0000","last_became_peered":"2024-04-15T15:18:44.869040+0000","last_unstale":"2024-04-15T15:18:46.818726+0000","last_undegraded":"2024-04-15T15:18:46.818726+0000","last_fullsized":"2024-04-15T15:18:46.818726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:54.111084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823654+0000","last_change":"2024-04-15T15:18:45.851111+0000","last_active":"2024-04-15T15:18:46.823654+0000","last_peered":"2024-04-15T15:18:46.823654+0000","last_clean":"2024-04-15T15:18:46.823654+0000","last_became_active":"2024-04-15T15:18:45.850824+0000","last_became_peered":"2024-04-15T15:18:45.850824+0000","last_unstale":"2024-04-15T15:18:46.823654+0000","last_undegraded":"2024-04-15T15:18:46.823654+0000","last_fullsized":"2024-04-15T15:18:46.823654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:53.984837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818964+0000","last_change":"2024-04-15T15:18:44.864267+0000","last_active":"2024-04-15T15:18:46.818964+0000","last_peered":"2024-04-15T15:18:46.818964+0000","last_clean":"2024-04-15T15:18:46.818964+0000","last_became_active":"2024-04-15T15:18:44.863962+0000","last_became_peered":"2024-04-15T15:18:44.863962+0000","last_unstale":"2024-04-15T15:18:46.818964+0000","last_undegraded":"2024-04-15T15:18:46.818964+0000","last_fullsized":"2024-04-15T15:18:46.818964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:22:53.739928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868055+0000","last_change":"2024-04-15T15:18:45.845359+0000","last_active":"2024-04-15T15:18:47.868055+0000","last_peered":"2024-04-15T15:18:47.868055+0000","last_clean":"2024-04-15T15:18:47.868055+0000","last_became_active":"2024-04-15T15:18:45.845087+0000","last_became_peered":"2024-04-15T15:18:45.845087+0000","last_unstale":"2024-04-15T15:18:47.868055+0000","last_undegraded":"2024-04-15T15:18:47.868055+0000","last_fullsized":"2024-04-15T15:18:47.868055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868578+0000","last_change":"2024-04-15T15:18:44.864497+0000","last_active":"2024-04-15T15:18:47.868578+0000","last_peered":"2024-04-15T15:18:47.868578+0000","last_clean":"2024-04-15T15:18:47.868578+0000","last_became_active":"2024-04-15T15:18:44.864122+0000","last_became_peered":"2024-04-15T15:18:44.864122+0000","last_unstale":"2024-04-15T15:18:47.868578+0000","last_undegraded":"2024-04-15T15:18:47.868578+0000","last_fullsized":"2024-04-15T15:18:47.868578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868175+0000","last_change":"2024-04-15T15:18:45.861978+0000","last_active":"2024-04-15T15:18:47.868175+0000","last_peered":"2024-04-15T15:18:47.868175+0000","last_clean":"2024-04-15T15:18:47.868175+0000","last_became_active":"2024-04-15T15:18:45.861597+0000","last_became_peered":"2024-04-15T15:18:45.861597+0000","last_unstale":"2024-04-15T15:18:47.868175+0000","last_undegraded":"2024-04-15T15:18:47.868175+0000","last_fullsized":"2024-04-15T15:18:47.868175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868895+0000","last_change":"2024-04-15T15:18:44.864325+0000","last_active":"2024-04-15T15:18:47.868895+0000","last_peered":"2024-04-15T15:18:47.868895+0000","last_clean":"2024-04-15T15:18:47.868895+0000","last_became_active":"2024-04-15T15:18:44.863985+0000","last_became_peered":"2024-04-15T15:18:44.863985+0000","last_unstale":"2024-04-15T15:18:47.868895+0000","last_undegraded":"2024-04-15T15:18:47.868895+0000","last_fullsized":"2024-04-15T15:18:47.868895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":823296,"data_stored":669858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":79034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":237568,"data_stored":80520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":78908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":79068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":831488,"data_stored":669914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:18:56.171 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:18:56.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.432+0000 7f0413200640 1 Processor -- start 2024-04-15T15:18:56.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.433+0000 7f0413200640 1 -- start start 2024-04-15T15:18:56.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.434+0000 7f0413200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f04140b3c70 0x7f04140d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.434+0000 7f0413200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04140d1af0 0x7f04140d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.434+0000 7f0413200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f041415ef90 con 0x7f041415ebb0 2024-04-15T15:18:56.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.434+0000 7f0413200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f04140d1f60 con 0x7f04140b3c70 2024-04-15T15:18:56.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.434+0000 7f0413200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f04140cf090 con 0x7f04140d1af0 2024-04-15T15:18:56.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.435+0000 7f0411e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f041415ebb0 0x7f0414163060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42928/0 (socket says 172.21.15.49:42928) 2024-04-15T15:18:56.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.435+0000 7f0411e00640 1 -- 172.21.15.49:0/3249042494 learned_addr learned my addr 172.21.15.49:0/3249042494 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:56.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.435+0000 7f040be00640 1 --2- 172.21.15.49:0/3249042494 >> v2:172.21.15.107:3300/0 conn(0x7f04140b3c70 0x7f04140d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.435+0000 7f0412800640 1 --2- 172.21.15.49:0/3249042494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04140d1af0 0x7f04140d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.435+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 929693701 0 0) 0x7f041415ef90 con 0x7f041415ebb0 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.435+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03f8003520 con 0x7f041415ebb0 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 947372480 0 0) 0x7f03f8003520 con 0x7f041415ebb0 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04140d1af0 msgr2=0x7f04140d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 --2- 172.21.15.49:0/3249042494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04140d1af0 0x7f04140d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 >> v2:172.21.15.107:3300/0 conn(0x7f04140b3c70 msgr2=0x7f04140d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 --2- 172.21.15.49:0/3249042494 >> v2:172.21.15.107:3300/0 conn(0x7f04140b3c70 0x7f04140d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04140cf370 con 0x7f041415ebb0 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f0404001670 con 0x7f041415ebb0 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 >> v1:172.21.15.49:6789/0 conn(0x7f041415ebb0 legacy=0x7f0414163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 --2- 172.21.15.49:0/3249042494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f80039b0 0x7f03f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 --2- 172.21.15.49:0/3249042494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03f8004800 0x7f03f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 --2- 172.21.15.49:0/3249042494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04140d1af0 0x7f03f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f04140d1f60 con 0x7f03f8004800 2024-04-15T15:18:56.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04140cf090 con 0x7f03f80039b0 2024-04-15T15:18:56.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f03f800b9c0 con 0x7f04140d1af0 2024-04-15T15:18:56.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.436+0000 7f040be00640 1 --2- 172.21.15.49:0/3249042494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03f8004800 0x7f03f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.437+0000 7f040be00640 1 -- 172.21.15.49:0/3249042494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04140d1af0 msgr2=0x7f03f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:56.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.437+0000 7f040be00640 1 --2- 172.21.15.49:0/3249042494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04140d1af0 0x7f03f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.437+0000 7f040be00640 1 -- 172.21.15.49:0/3249042494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f80039b0 msgr2=0x7f03f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:56.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.437+0000 7f040be00640 1 --2- 172.21.15.49:0/3249042494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f80039b0 0x7f03f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.437+0000 7f040be00640 1 -- 172.21.15.49:0/3249042494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f8003520 con 0x7f03f8004800 2024-04-15T15:18:56.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.438+0000 7f040be00640 1 --2- 172.21.15.49:0/3249042494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03f8004800 0x7f03f8004c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009ac0 tx=0x7f03f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccde684107c8a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:56.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.438+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc039070 con 0x7f03f8004800 2024-04-15T15:18:56.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.438+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03fc02c6d0 con 0x7f03f8004800 2024-04-15T15:18:56.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.438+0000 7f0411400640 1 -- 172.21.15.49:0/3249042494 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc009690 con 0x7f03f8004800 2024-04-15T15:18:56.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.438+0000 7f0413200640 1 -- 172.21.15.49:0/3249042494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03f8004800 msgr2=0x7f03f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.438+0000 7f0413200640 1 --2- 172.21.15.49:0/3249042494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03f8004800 0x7f03f8004c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009ac0 tx=0x7f03f8005830 comp rx=0 tx=0).stop 2024-04-15T15:18:56.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.438+0000 7f0412800640 1 -- 172.21.15.49:0/3249042494 reap_dead start 2024-04-15T15:18:56.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.439+0000 7f0413200640 1 -- 172.21.15.49:0/3249042494 shutdown_connections 2024-04-15T15:18:56.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.439+0000 7f0413200640 1 -- 172.21.15.49:0/3249042494 >> 172.21.15.49:0/3249042494 conn(0x7f041406d040 msgr2=0x7f041406d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:56.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.439+0000 7f0413200640 1 -- 172.21.15.49:0/3249042494 shutdown_connections 2024-04-15T15:18:56.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.439+0000 7f0413200640 1 -- 172.21.15.49:0/3249042494 wait complete. 2024-04-15T15:18:56.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.440+0000 7f0413200640 1 Processor -- start 2024-04-15T15:18:56.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.441+0000 7f0413200640 1 -- start start 2024-04-15T15:18:56.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f0413200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03f8004800 0x7f041415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f0413200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04140b8f50 0x7f04140b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f0413200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04141f4d40 0x7f04141f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f0413200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f04140cfc10 con 0x7f04140b8f50 2024-04-15T15:18:56.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f0413200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04140cf930 con 0x7f04141f4d40 2024-04-15T15:18:56.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f0413200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f04140cfef0 con 0x7f03f8004800 2024-04-15T15:18:56.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f040be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04140b8f50 0x7f04140b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f040be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04140b8f50 0x7f04140b93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39910/0 (socket says 172.21.15.49:39910) 2024-04-15T15:18:56.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f040be00640 1 -- 172.21.15.49:0/3037699712 learned_addr learned my addr 172.21.15.49:0/3037699712 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:56.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f0411e00640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03f8004800 0x7f041415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f040be00640 1 -- 172.21.15.49:0/3037699712 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03f8004800 msgr2=0x7f041415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f040be00640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03f8004800 0x7f041415bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f0412800640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04141f4d40 0x7f04141f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f040be00640 1 -- 172.21.15.49:0/3037699712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04141f4d40 msgr2=0x7f04141f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f040be00640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04141f4d40 0x7f04141f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.442+0000 7f040be00640 1 -- 172.21.15.49:0/3037699712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04140acac0 con 0x7f04140b8f50 2024-04-15T15:18:56.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.443+0000 7f040be00640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04140b8f50 0x7f04140b93b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f03fc000c00 tx=0x7f03fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:56.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.443+0000 7f040a000640 1 -- 172.21.15.49:0/3037699712 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc039050 con 0x7f04140b8f50 2024-04-15T15:18:56.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.443+0000 7f040a000640 1 -- 172.21.15.49:0/3037699712 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03fc045070 con 0x7f04140b8f50 2024-04-15T15:18:56.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.443+0000 7f040a000640 1 -- 172.21.15.49:0/3037699712 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc003a50 con 0x7f04140b8f50 2024-04-15T15:18:56.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.443+0000 7f0413200640 1 -- 172.21.15.49:0/3037699712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04140abd00 con 0x7f04140b8f50 2024-04-15T15:18:56.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.443+0000 7f0413200640 1 -- 172.21.15.49:0/3037699712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04140ac2c0 con 0x7f04140b8f50 2024-04-15T15:18:56.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.444+0000 7f0413200640 1 -- 172.21.15.49:0/3037699712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04140755c0 con 0x7f04140b8f50 2024-04-15T15:18:56.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.445+0000 7f040a000640 1 -- 172.21.15.49:0/3037699712 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03fc003bf0 con 0x7f04140b8f50 2024-04-15T15:18:56.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.446+0000 7f040a000640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f03f006c920 0x7f03f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.447+0000 7f040a000640 1 -- 172.21.15.49:0/3037699712 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f03fc0ad650 con 0x7f04140b8f50 2024-04-15T15:18:56.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.447+0000 7f0411e00640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f03f006c920 0x7f03f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.447+0000 7f0411e00640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f03f006c920 0x7f03f006ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0404001eb0 tx=0x7f0404001730 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.451+0000 7f040a000640 1 -- 172.21.15.49:0/3037699712 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03fc04e020 con 0x7f04140b8f50 2024-04-15T15:18:56.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.644+0000 7f0413200640 1 -- 172.21.15.49:0/3037699712 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04140abab0 con 0x7f03f006c920 2024-04-15T15:18:56.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.664+0000 7f040a000640 1 -- 172.21.15.49:0/3037699712 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251839 (secure 0 0 0) 0x7f04140abab0 con 0x7f03f006c920 2024-04-15T15:18:56.672 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:56.673 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:18:56.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.669+0000 7f0413200640 1 -- 172.21.15.49:0/3037699712 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f03f006c920 msgr2=0x7f03f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.669+0000 7f0413200640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f03f006c920 0x7f03f006ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0404001eb0 tx=0x7f0404001730 comp rx=0 tx=0).stop 2024-04-15T15:18:56.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.669+0000 7f0413200640 1 -- 172.21.15.49:0/3037699712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04140b8f50 msgr2=0x7f04140b93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.669+0000 7f0413200640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04140b8f50 0x7f04140b93b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f03fc000c00 tx=0x7f03fc004730 comp rx=0 tx=0).stop 2024-04-15T15:18:56.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.670+0000 7f0413200640 1 -- 172.21.15.49:0/3037699712 shutdown_connections 2024-04-15T15:18:56.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.670+0000 7f0413200640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f03f006c920 0x7f03f006ede0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.670+0000 7f0413200640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04141f4d40 0x7f04141f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.670+0000 7f0413200640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04140b8f50 0x7f04140b93b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.670+0000 7f0413200640 1 --2- 172.21.15.49:0/3037699712 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03f8004800 0x7f041415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.670+0000 7f0413200640 1 -- 172.21.15.49:0/3037699712 >> 172.21.15.49:0/3037699712 conn(0x7f041406d040 msgr2=0x7f041415dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:56.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.670+0000 7f0413200640 1 -- 172.21.15.49:0/3037699712 shutdown_connections 2024-04-15T15:18:56.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.670+0000 7f0413200640 1 -- 172.21.15.49:0/3037699712 wait complete. 2024-04-15T15:18:56.691 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-15T15:18:54.773844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235952,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738768,"statfs":{"total":773094113280,"available":772852498432,"internally_reserved":0,"allocated":3055616,"data_stored":1816820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":62,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.964382"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.825719+0000","last_change":"2024-04-15T15:18:45.850244+0000","last_active":"2024-04-15T15:18:46.825719+0000","last_peered":"2024-04-15T15:18:46.825719+0000","last_clean":"2024-04-15T15:18:46.825719+0000","last_became_active":"2024-04-15T15:18:45.849857+0000","last_became_peered":"2024-04-15T15:18:45.849857+0000","last_unstale":"2024-04-15T15:18:46.825719+0000","last_undegraded":"2024-04-15T15:18:46.825719+0000","last_fullsized":"2024-04-15T15:18:46.825719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:08:04.293347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870331+0000","last_change":"2024-04-15T15:18:45.843425+0000","last_active":"2024-04-15T15:18:46.870331+0000","last_peered":"2024-04-15T15:18:46.870331+0000","last_clean":"2024-04-15T15:18:46.870331+0000","last_became_active":"2024-04-15T15:18:45.843052+0000","last_became_peered":"2024-04-15T15:18:45.843052+0000","last_unstale":"2024-04-15T15:18:46.870331+0000","last_undegraded":"2024-04-15T15:18:46.870331+0000","last_fullsized":"2024-04-15T15:18:46.870331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868186+0000","last_change":"2024-04-15T15:18:45.853920+0000","last_active":"2024-04-15T15:18:47.868186+0000","last_peered":"2024-04-15T15:18:47.868186+0000","last_clean":"2024-04-15T15:18:47.868186+0000","last_became_active":"2024-04-15T15:18:45.853659+0000","last_became_peered":"2024-04-15T15:18:45.853659+0000","last_unstale":"2024-04-15T15:18:47.868186+0000","last_undegraded":"2024-04-15T15:18:47.868186+0000","last_fullsized":"2024-04-15T15:18:47.868186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823670+0000","last_change":"2024-04-15T15:18:45.861695+0000","last_active":"2024-04-15T15:18:46.823670+0000","last_peered":"2024-04-15T15:18:46.823670+0000","last_clean":"2024-04-15T15:18:46.823670+0000","last_became_active":"2024-04-15T15:18:45.858560+0000","last_became_peered":"2024-04-15T15:18:45.858560+0000","last_unstale":"2024-04-15T15:18:46.823670+0000","last_undegraded":"2024-04-15T15:18:46.823670+0000","last_fullsized":"2024-04-15T15:18:46.823670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:29:14.132066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820061+0000","last_change":"2024-04-15T15:18:45.849530+0000","last_active":"2024-04-15T15:18:46.820061+0000","last_peered":"2024-04-15T15:18:46.820061+0000","last_clean":"2024-04-15T15:18:46.820061+0000","last_became_active":"2024-04-15T15:18:45.849255+0000","last_became_peered":"2024-04-15T15:18:45.849255+0000","last_unstale":"2024-04-15T15:18:46.820061+0000","last_undegraded":"2024-04-15T15:18:46.820061+0000","last_fullsized":"2024-04-15T15:18:46.820061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:36:03.614062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822193+0000","last_change":"2024-04-15T15:18:45.842283+0000","last_active":"2024-04-15T15:18:46.822193+0000","last_peered":"2024-04-15T15:18:46.822193+0000","last_clean":"2024-04-15T15:18:46.822193+0000","last_became_active":"2024-04-15T15:18:45.842004+0000","last_became_peered":"2024-04-15T15:18:45.842004+0000","last_unstale":"2024-04-15T15:18:46.822193+0000","last_undegraded":"2024-04-15T15:18:46.822193+0000","last_fullsized":"2024-04-15T15:18:46.822193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:08:29.977812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818174+0000","last_change":"2024-04-15T15:18:45.856124+0000","last_active":"2024-04-15T15:18:46.818174+0000","last_peered":"2024-04-15T15:18:46.818174+0000","last_clean":"2024-04-15T15:18:46.818174+0000","last_became_active":"2024-04-15T15:18:45.849034+0000","last_became_peered":"2024-04-15T15:18:45.849034+0000","last_unstale":"2024-04-15T15:18:46.818174+0000","last_undegraded":"2024-04-15T15:18:46.818174+0000","last_fullsized":"2024-04-15T15:18:46.818174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:49:43.342170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.866494+0000","last_change":"2024-04-15T15:18:45.848306+0000","last_active":"2024-04-15T15:18:47.866494+0000","last_peered":"2024-04-15T15:18:47.866494+0000","last_clean":"2024-04-15T15:18:47.866494+0000","last_became_active":"2024-04-15T15:18:45.848033+0000","last_became_peered":"2024-04-15T15:18:45.848033+0000","last_unstale":"2024-04-15T15:18:47.866494+0000","last_undegraded":"2024-04-15T15:18:47.866494+0000","last_fullsized":"2024-04-15T15:18:47.866494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.826189+0000","last_change":"2024-04-15T15:18:45.862452+0000","last_active":"2024-04-15T15:18:46.826189+0000","last_peered":"2024-04-15T15:18:46.826189+0000","last_clean":"2024-04-15T15:18:46.826189+0000","last_became_active":"2024-04-15T15:18:45.862157+0000","last_became_peered":"2024-04-15T15:18:45.862157+0000","last_unstale":"2024-04-15T15:18:46.826189+0000","last_undegraded":"2024-04-15T15:18:46.826189+0000","last_fullsized":"2024-04-15T15:18:46.826189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:14:17.068828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875290+0000","last_change":"2024-04-15T15:18:45.855709+0000","last_active":"2024-04-15T15:18:46.875290+0000","last_peered":"2024-04-15T15:18:46.875290+0000","last_clean":"2024-04-15T15:18:46.875290+0000","last_became_active":"2024-04-15T15:18:45.855243+0000","last_became_peered":"2024-04-15T15:18:45.855243+0000","last_unstale":"2024-04-15T15:18:46.875290+0000","last_undegraded":"2024-04-15T15:18:46.875290+0000","last_fullsized":"2024-04-15T15:18:46.875290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874937+0000","last_change":"2024-04-15T15:18:45.859684+0000","last_active":"2024-04-15T15:18:46.874937+0000","last_peered":"2024-04-15T15:18:46.874937+0000","last_clean":"2024-04-15T15:18:46.874937+0000","last_became_active":"2024-04-15T15:18:45.856683+0000","last_became_peered":"2024-04-15T15:18:45.856683+0000","last_unstale":"2024-04-15T15:18:46.874937+0000","last_undegraded":"2024-04-15T15:18:46.874937+0000","last_fullsized":"2024-04-15T15:18:46.874937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822415+0000","last_change":"2024-04-15T15:18:45.849769+0000","last_active":"2024-04-15T15:18:46.822415+0000","last_peered":"2024-04-15T15:18:46.822415+0000","last_clean":"2024-04-15T15:18:46.822415+0000","last_became_active":"2024-04-15T15:18:45.849462+0000","last_became_peered":"2024-04-15T15:18:45.849462+0000","last_unstale":"2024-04-15T15:18:46.822415+0000","last_undegraded":"2024-04-15T15:18:46.822415+0000","last_fullsized":"2024-04-15T15:18:46.822415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:35:53.211120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868757+0000","last_change":"2024-04-15T15:18:45.852551+0000","last_active":"2024-04-15T15:18:47.868757+0000","last_peered":"2024-04-15T15:18:47.868757+0000","last_clean":"2024-04-15T15:18:47.868757+0000","last_became_active":"2024-04-15T15:18:45.851658+0000","last_became_peered":"2024-04-15T15:18:45.851658+0000","last_unstale":"2024-04-15T15:18:47.868757+0000","last_undegraded":"2024-04-15T15:18:47.868757+0000","last_fullsized":"2024-04-15T15:18:47.868757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:49:43.342170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.866096+0000","last_change":"2024-04-15T15:18:45.842127+0000","last_active":"2024-04-15T15:18:47.866096+0000","last_peered":"2024-04-15T15:18:47.866096+0000","last_clean":"2024-04-15T15:18:47.866096+0000","last_became_active":"2024-04-15T15:18:45.841662+0000","last_became_peered":"2024-04-15T15:18:45.841662+0000","last_unstale":"2024-04-15T15:18:47.866096+0000","last_undegraded":"2024-04-15T15:18:47.866096+0000","last_fullsized":"2024-04-15T15:18:47.866096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:39:00.386699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869891+0000","last_change":"2024-04-15T15:18:45.832097+0000","last_active":"2024-04-15T15:18:46.869891+0000","last_peered":"2024-04-15T15:18:46.869891+0000","last_clean":"2024-04-15T15:18:46.869891+0000","last_became_active":"2024-04-15T15:18:45.831411+0000","last_became_peered":"2024-04-15T15:18:45.831411+0000","last_unstale":"2024-04-15T15:18:46.869891+0000","last_undegraded":"2024-04-15T15:18:46.869891+0000","last_fullsized":"2024-04-15T15:18:46.869891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822037+0000","last_change":"2024-04-15T15:18:45.846022+0000","last_active":"2024-04-15T15:18:46.822037+0000","last_peered":"2024-04-15T15:18:46.822037+0000","last_clean":"2024-04-15T15:18:46.822037+0000","last_became_active":"2024-04-15T15:18:45.844025+0000","last_became_peered":"2024-04-15T15:18:45.844025+0000","last_unstale":"2024-04-15T15:18:46.822037+0000","last_undegraded":"2024-04-15T15:18:46.822037+0000","last_fullsized":"2024-04-15T15:18:46.822037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.825219+0000","last_change":"2024-04-15T15:18:45.849397+0000","last_active":"2024-04-15T15:18:46.825219+0000","last_peered":"2024-04-15T15:18:46.825219+0000","last_clean":"2024-04-15T15:18:46.825219+0000","last_became_active":"2024-04-15T15:18:45.849114+0000","last_became_peered":"2024-04-15T15:18:45.849114+0000","last_unstale":"2024-04-15T15:18:46.825219+0000","last_undegraded":"2024-04-15T15:18:46.825219+0000","last_fullsized":"2024-04-15T15:18:46.825219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:08:04.293347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.864639+0000","last_change":"2024-04-15T15:18:45.848118+0000","last_active":"2024-04-15T15:18:47.864639+0000","last_peered":"2024-04-15T15:18:47.864639+0000","last_clean":"2024-04-15T15:18:47.864639+0000","last_became_active":"2024-04-15T15:18:45.847644+0000","last_became_peered":"2024-04-15T15:18:45.847644+0000","last_unstale":"2024-04-15T15:18:47.864639+0000","last_undegraded":"2024-04-15T15:18:47.864639+0000","last_fullsized":"2024-04-15T15:18:47.864639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:22:54.875123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820072+0000","last_change":"2024-04-15T15:18:45.860524+0000","last_active":"2024-04-15T15:18:46.820072+0000","last_peered":"2024-04-15T15:18:46.820072+0000","last_clean":"2024-04-15T15:18:46.820072+0000","last_became_active":"2024-04-15T15:18:45.852020+0000","last_became_peered":"2024-04-15T15:18:45.852020+0000","last_unstale":"2024-04-15T15:18:46.820072+0000","last_undegraded":"2024-04-15T15:18:46.820072+0000","last_fullsized":"2024-04-15T15:18:46.820072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:06:25.954239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874881+0000","last_change":"2024-04-15T15:18:45.852135+0000","last_active":"2024-04-15T15:18:46.874881+0000","last_peered":"2024-04-15T15:18:46.874881+0000","last_clean":"2024-04-15T15:18:46.874881+0000","last_became_active":"2024-04-15T15:18:45.851841+0000","last_became_peered":"2024-04-15T15:18:45.851841+0000","last_unstale":"2024-04-15T15:18:46.874881+0000","last_undegraded":"2024-04-15T15:18:46.874881+0000","last_fullsized":"2024-04-15T15:18:46.874881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868076+0000","last_change":"2024-04-15T15:18:45.859189+0000","last_active":"2024-04-15T15:18:47.868076+0000","last_peered":"2024-04-15T15:18:47.868076+0000","last_clean":"2024-04-15T15:18:47.868076+0000","last_became_active":"2024-04-15T15:18:45.857117+0000","last_became_peered":"2024-04-15T15:18:45.857117+0000","last_unstale":"2024-04-15T15:18:47.868076+0000","last_undegraded":"2024-04-15T15:18:47.868076+0000","last_fullsized":"2024-04-15T15:18:47.868076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822607+0000","last_change":"2024-04-15T15:18:45.840913+0000","last_active":"2024-04-15T15:18:46.822607+0000","last_peered":"2024-04-15T15:18:46.822607+0000","last_clean":"2024-04-15T15:18:46.822607+0000","last_became_active":"2024-04-15T15:18:45.840257+0000","last_became_peered":"2024-04-15T15:18:45.840257+0000","last_unstale":"2024-04-15T15:18:46.822607+0000","last_undegraded":"2024-04-15T15:18:46.822607+0000","last_fullsized":"2024-04-15T15:18:46.822607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:49:43.342170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.867493+0000","last_change":"2024-04-15T15:18:45.851064+0000","last_active":"2024-04-15T15:18:47.867493+0000","last_peered":"2024-04-15T15:18:47.867493+0000","last_clean":"2024-04-15T15:18:47.867493+0000","last_became_active":"2024-04-15T15:18:45.850725+0000","last_became_peered":"2024-04-15T15:18:45.850725+0000","last_unstale":"2024-04-15T15:18:47.867493+0000","last_undegraded":"2024-04-15T15:18:47.867493+0000","last_fullsized":"2024-04-15T15:18:47.867493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820869+0000","last_change":"2024-04-15T15:18:45.838476+0000","last_active":"2024-04-15T15:18:46.820869+0000","last_peered":"2024-04-15T15:18:46.820869+0000","last_clean":"2024-04-15T15:18:46.820869+0000","last_became_active":"2024-04-15T15:18:45.838160+0000","last_became_peered":"2024-04-15T15:18:45.838160+0000","last_unstale":"2024-04-15T15:18:46.820869+0000","last_undegraded":"2024-04-15T15:18:46.820869+0000","last_fullsized":"2024-04-15T15:18:46.820869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868577+0000","last_change":"2024-04-15T15:18:45.855537+0000","last_active":"2024-04-15T15:18:47.868577+0000","last_peered":"2024-04-15T15:18:47.868577+0000","last_clean":"2024-04-15T15:18:47.868577+0000","last_became_active":"2024-04-15T15:18:45.855144+0000","last_became_peered":"2024-04-15T15:18:45.855144+0000","last_unstale":"2024-04-15T15:18:47.868577+0000","last_undegraded":"2024-04-15T15:18:47.868577+0000","last_fullsized":"2024-04-15T15:18:47.868577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875245+0000","last_change":"2024-04-15T15:18:45.848420+0000","last_active":"2024-04-15T15:18:46.875245+0000","last_peered":"2024-04-15T15:18:46.875245+0000","last_clean":"2024-04-15T15:18:46.875245+0000","last_became_active":"2024-04-15T15:18:45.848080+0000","last_became_peered":"2024-04-15T15:18:45.848080+0000","last_unstale":"2024-04-15T15:18:46.875245+0000","last_undegraded":"2024-04-15T15:18:46.875245+0000","last_fullsized":"2024-04-15T15:18:46.875245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869996+0000","last_change":"2024-04-15T15:18:45.858358+0000","last_active":"2024-04-15T15:18:46.869996+0000","last_peered":"2024-04-15T15:18:46.869996+0000","last_clean":"2024-04-15T15:18:46.869996+0000","last_became_active":"2024-04-15T15:18:45.856289+0000","last_became_peered":"2024-04-15T15:18:45.856289+0000","last_unstale":"2024-04-15T15:18:46.869996+0000","last_undegraded":"2024-04-15T15:18:46.869996+0000","last_fullsized":"2024-04-15T15:18:46.869996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868753+0000","last_change":"2024-04-15T15:18:45.854349+0000","last_active":"2024-04-15T15:18:47.868753+0000","last_peered":"2024-04-15T15:18:47.868753+0000","last_clean":"2024-04-15T15:18:47.868753+0000","last_became_active":"2024-04-15T15:18:45.853978+0000","last_became_peered":"2024-04-15T15:18:45.853978+0000","last_unstale":"2024-04-15T15:18:47.868753+0000","last_undegraded":"2024-04-15T15:18:47.868753+0000","last_fullsized":"2024-04-15T15:18:47.868753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:08:29.977812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.867237+0000","last_change":"2024-04-15T15:18:45.858661+0000","last_active":"2024-04-15T15:18:47.867237+0000","last_peered":"2024-04-15T15:18:47.867237+0000","last_clean":"2024-04-15T15:18:47.867237+0000","last_became_active":"2024-04-15T15:18:45.858244+0000","last_became_peered":"2024-04-15T15:18:45.858244+0000","last_unstale":"2024-04-15T15:18:47.867237+0000","last_undegraded":"2024-04-15T15:18:47.867237+0000","last_fullsized":"2024-04-15T15:18:47.867237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823029+0000","last_change":"2024-04-15T15:18:45.840590+0000","last_active":"2024-04-15T15:18:46.823029+0000","last_peered":"2024-04-15T15:18:46.823029+0000","last_clean":"2024-04-15T15:18:46.823029+0000","last_became_active":"2024-04-15T15:18:45.839849+0000","last_became_peered":"2024-04-15T15:18:45.839849+0000","last_unstale":"2024-04-15T15:18:46.823029+0000","last_undegraded":"2024-04-15T15:18:46.823029+0000","last_fullsized":"2024-04-15T15:18:46.823029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.566254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.864522+0000","last_change":"2024-04-15T15:18:45.847940+0000","last_active":"2024-04-15T15:18:47.864522+0000","last_peered":"2024-04-15T15:18:47.864522+0000","last_clean":"2024-04-15T15:18:47.864522+0000","last_became_active":"2024-04-15T15:18:45.846916+0000","last_became_peered":"2024-04-15T15:18:45.846916+0000","last_unstale":"2024-04-15T15:18:47.864522+0000","last_undegraded":"2024-04-15T15:18:47.864522+0000","last_fullsized":"2024-04-15T15:18:47.864522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:27.690982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874483+0000","last_change":"2024-04-15T15:18:45.846327+0000","last_active":"2024-04-15T15:18:46.874483+0000","last_peered":"2024-04-15T15:18:46.874483+0000","last_clean":"2024-04-15T15:18:46.874483+0000","last_became_active":"2024-04-15T15:18:45.846010+0000","last_became_peered":"2024-04-15T15:18:45.846010+0000","last_unstale":"2024-04-15T15:18:46.874483+0000","last_undegraded":"2024-04-15T15:18:46.874483+0000","last_fullsized":"2024-04-15T15:18:46.874483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820924+0000","last_change":"2024-04-15T15:18:45.846764+0000","last_active":"2024-04-15T15:18:46.820924+0000","last_peered":"2024-04-15T15:18:46.820924+0000","last_clean":"2024-04-15T15:18:46.820924+0000","last_became_active":"2024-04-15T15:18:45.842870+0000","last_became_peered":"2024-04-15T15:18:45.842870+0000","last_unstale":"2024-04-15T15:18:46.820924+0000","last_undegraded":"2024-04-15T15:18:46.820924+0000","last_fullsized":"2024-04-15T15:18:46.820924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:38:23.700029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.869023+0000","last_change":"2024-04-15T15:18:44.864457+0000","last_active":"2024-04-15T15:18:47.869023+0000","last_peered":"2024-04-15T15:18:47.869023+0000","last_clean":"2024-04-15T15:18:47.869023+0000","last_became_active":"2024-04-15T15:18:44.863999+0000","last_became_peered":"2024-04-15T15:18:44.863999+0000","last_unstale":"2024-04-15T15:18:47.869023+0000","last_undegraded":"2024-04-15T15:18:47.869023+0000","last_fullsized":"2024-04-15T15:18:47.869023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.867390+0000","last_change":"2024-04-15T15:18:45.855613+0000","last_active":"2024-04-15T15:18:47.867390+0000","last_peered":"2024-04-15T15:18:47.867390+0000","last_clean":"2024-04-15T15:18:47.867390+0000","last_became_active":"2024-04-15T15:18:45.855327+0000","last_became_peered":"2024-04-15T15:18:45.855327+0000","last_unstale":"2024-04-15T15:18:47.867390+0000","last_undegraded":"2024-04-15T15:18:47.867390+0000","last_fullsized":"2024-04-15T15:18:47.867390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874502+0000","last_change":"2024-04-15T15:18:44.864183+0000","last_active":"2024-04-15T15:18:46.874502+0000","last_peered":"2024-04-15T15:18:46.874502+0000","last_clean":"2024-04-15T15:18:46.874502+0000","last_became_active":"2024-04-15T15:18:44.861740+0000","last_became_peered":"2024-04-15T15:18:44.861740+0000","last_unstale":"2024-04-15T15:18:46.874502+0000","last_undegraded":"2024-04-15T15:18:46.874502+0000","last_fullsized":"2024-04-15T15:18:46.874502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.867367+0000","last_change":"2024-04-15T15:18:44.867636+0000","last_active":"2024-04-15T15:18:47.867367+0000","last_peered":"2024-04-15T15:18:47.867367+0000","last_clean":"2024-04-15T15:18:47.867367+0000","last_became_active":"2024-04-15T15:18:44.867307+0000","last_became_peered":"2024-04-15T15:18:44.867307+0000","last_unstale":"2024-04-15T15:18:47.867367+0000","last_undegraded":"2024-04-15T15:18:47.867367+0000","last_fullsized":"2024-04-15T15:18:47.867367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.865159+0000","last_change":"2024-04-15T15:18:45.850873+0000","last_active":"2024-04-15T15:18:47.865159+0000","last_peered":"2024-04-15T15:18:47.865159+0000","last_clean":"2024-04-15T15:18:47.865159+0000","last_became_active":"2024-04-15T15:18:45.850532+0000","last_became_peered":"2024-04-15T15:18:45.850532+0000","last_unstale":"2024-04-15T15:18:47.865159+0000","last_undegraded":"2024-04-15T15:18:47.865159+0000","last_fullsized":"2024-04-15T15:18:47.865159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821134+0000","last_change":"2024-04-15T15:18:44.866286+0000","last_active":"2024-04-15T15:18:46.821134+0000","last_peered":"2024-04-15T15:18:46.821134+0000","last_clean":"2024-04-15T15:18:46.821134+0000","last_became_active":"2024-04-15T15:18:44.863387+0000","last_became_peered":"2024-04-15T15:18:44.863387+0000","last_unstale":"2024-04-15T15:18:46.821134+0000","last_undegraded":"2024-04-15T15:18:46.821134+0000","last_fullsized":"2024-04-15T15:18:46.821134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869851+0000","last_change":"2024-04-15T15:18:45.839256+0000","last_active":"2024-04-15T15:18:46.869851+0000","last_peered":"2024-04-15T15:18:46.869851+0000","last_clean":"2024-04-15T15:18:46.869851+0000","last_became_active":"2024-04-15T15:18:45.838960+0000","last_became_peered":"2024-04-15T15:18:45.838960+0000","last_unstale":"2024-04-15T15:18:46.869851+0000","last_undegraded":"2024-04-15T15:18:46.869851+0000","last_fullsized":"2024-04-15T15:18:46.869851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875122+0000","last_change":"2024-04-15T15:18:44.870228+0000","last_active":"2024-04-15T15:18:46.875122+0000","last_peered":"2024-04-15T15:18:46.875122+0000","last_clean":"2024-04-15T15:18:46.875122+0000","last_became_active":"2024-04-15T15:18:44.869155+0000","last_became_peered":"2024-04-15T15:18:44.869155+0000","last_unstale":"2024-04-15T15:18:46.875122+0000","last_undegraded":"2024-04-15T15:18:46.875122+0000","last_fullsized":"2024-04-15T15:18:46.875122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824756+0000","last_change":"2024-04-15T15:18:45.845321+0000","last_active":"2024-04-15T15:18:46.824756+0000","last_peered":"2024-04-15T15:18:46.824756+0000","last_clean":"2024-04-15T15:18:46.824756+0000","last_became_active":"2024-04-15T15:18:45.840422+0000","last_became_peered":"2024-04-15T15:18:45.840422+0000","last_unstale":"2024-04-15T15:18:46.824756+0000","last_undegraded":"2024-04-15T15:18:46.824756+0000","last_fullsized":"2024-04-15T15:18:46.824756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:32:58.852361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824852+0000","last_change":"2024-04-15T15:18:44.865670+0000","last_active":"2024-04-15T15:18:46.824852+0000","last_peered":"2024-04-15T15:18:46.824852+0000","last_clean":"2024-04-15T15:18:46.824852+0000","last_became_active":"2024-04-15T15:18:44.864795+0000","last_became_peered":"2024-04-15T15:18:44.864795+0000","last_unstale":"2024-04-15T15:18:46.824852+0000","last_undegraded":"2024-04-15T15:18:46.824852+0000","last_fullsized":"2024-04-15T15:18:46.824852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.869467+0000","last_change":"2024-04-15T15:18:45.852169+0000","last_active":"2024-04-15T15:18:47.869467+0000","last_peered":"2024-04-15T15:18:47.869467+0000","last_clean":"2024-04-15T15:18:47.869467+0000","last_became_active":"2024-04-15T15:18:45.845297+0000","last_became_peered":"2024-04-15T15:18:45.845297+0000","last_unstale":"2024-04-15T15:18:47.869467+0000","last_undegraded":"2024-04-15T15:18:47.869467+0000","last_fullsized":"2024-04-15T15:18:47.869467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:08:29.977812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870182+0000","last_change":"2024-04-15T15:18:44.866158+0000","last_active":"2024-04-15T15:18:46.870182+0000","last_peered":"2024-04-15T15:18:46.870182+0000","last_clean":"2024-04-15T15:18:46.870182+0000","last_became_active":"2024-04-15T15:18:44.865766+0000","last_became_peered":"2024-04-15T15:18:44.865766+0000","last_unstale":"2024-04-15T15:18:46.870182+0000","last_undegraded":"2024-04-15T15:18:46.870182+0000","last_fullsized":"2024-04-15T15:18:46.870182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822687+0000","last_change":"2024-04-15T15:18:45.843966+0000","last_active":"2024-04-15T15:18:46.822687+0000","last_peered":"2024-04-15T15:18:46.822687+0000","last_clean":"2024-04-15T15:18:46.822687+0000","last_became_active":"2024-04-15T15:18:45.842648+0000","last_became_peered":"2024-04-15T15:18:45.842648+0000","last_unstale":"2024-04-15T15:18:46.822687+0000","last_undegraded":"2024-04-15T15:18:46.822687+0000","last_fullsized":"2024-04-15T15:18:46.822687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:39:50.034981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874917+0000","last_change":"2024-04-15T15:18:44.867158+0000","last_active":"2024-04-15T15:18:46.874917+0000","last_peered":"2024-04-15T15:18:46.874917+0000","last_clean":"2024-04-15T15:18:46.874917+0000","last_became_active":"2024-04-15T15:18:44.866840+0000","last_became_peered":"2024-04-15T15:18:44.866840+0000","last_unstale":"2024-04-15T15:18:46.874917+0000","last_undegraded":"2024-04-15T15:18:46.874917+0000","last_fullsized":"2024-04-15T15:18:46.874917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818675+0000","last_change":"2024-04-15T15:18:45.860752+0000","last_active":"2024-04-15T15:18:46.818675+0000","last_peered":"2024-04-15T15:18:46.818675+0000","last_clean":"2024-04-15T15:18:46.818675+0000","last_became_active":"2024-04-15T15:18:45.860381+0000","last_became_peered":"2024-04-15T15:18:45.860381+0000","last_unstale":"2024-04-15T15:18:46.818675+0000","last_undegraded":"2024-04-15T15:18:46.818675+0000","last_fullsized":"2024-04-15T15:18:46.818675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:53.984837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.867491+0000","last_change":"2024-04-15T15:18:44.865018+0000","last_active":"2024-04-15T15:18:47.867491+0000","last_peered":"2024-04-15T15:18:47.867491+0000","last_clean":"2024-04-15T15:18:47.867491+0000","last_became_active":"2024-04-15T15:18:44.864073+0000","last_became_peered":"2024-04-15T15:18:44.864073+0000","last_unstale":"2024-04-15T15:18:47.867491+0000","last_undegraded":"2024-04-15T15:18:47.867491+0000","last_fullsized":"2024-04-15T15:18:47.867491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:34.796362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.866527+0000","last_change":"2024-04-15T15:18:45.860189+0000","last_active":"2024-04-15T15:18:47.866527+0000","last_peered":"2024-04-15T15:18:47.866527+0000","last_clean":"2024-04-15T15:18:47.866527+0000","last_became_active":"2024-04-15T15:18:45.859786+0000","last_became_peered":"2024-04-15T15:18:45.859786+0000","last_unstale":"2024-04-15T15:18:47.866527+0000","last_undegraded":"2024-04-15T15:18:47.866527+0000","last_fullsized":"2024-04-15T15:18:47.866527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822942+0000","last_change":"2024-04-15T15:18:44.866125+0000","last_active":"2024-04-15T15:18:46.822942+0000","last_peered":"2024-04-15T15:18:46.822942+0000","last_clean":"2024-04-15T15:18:46.822942+0000","last_became_active":"2024-04-15T15:18:44.864849+0000","last_became_peered":"2024-04-15T15:18:44.864849+0000","last_unstale":"2024-04-15T15:18:46.822942+0000","last_undegraded":"2024-04-15T15:18:46.822942+0000","last_fullsized":"2024-04-15T15:18:46.822942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819575+0000","last_change":"2024-04-15T15:18:45.852862+0000","last_active":"2024-04-15T15:18:46.819575+0000","last_peered":"2024-04-15T15:18:46.819575+0000","last_clean":"2024-04-15T15:18:46.819575+0000","last_became_active":"2024-04-15T15:18:45.851610+0000","last_became_peered":"2024-04-15T15:18:45.851610+0000","last_unstale":"2024-04-15T15:18:46.819575+0000","last_undegraded":"2024-04-15T15:18:46.819575+0000","last_fullsized":"2024-04-15T15:18:46.819575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:36.969295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874519+0000","last_change":"2024-04-15T15:18:44.871366+0000","last_active":"2024-04-15T15:18:46.874519+0000","last_peered":"2024-04-15T15:18:46.874519+0000","last_clean":"2024-04-15T15:18:46.874519+0000","last_became_active":"2024-04-15T15:18:44.870865+0000","last_became_peered":"2024-04-15T15:18:44.870865+0000","last_unstale":"2024-04-15T15:18:46.874519+0000","last_undegraded":"2024-04-15T15:18:46.874519+0000","last_fullsized":"2024-04-15T15:18:46.874519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874458+0000","last_change":"2024-04-15T15:18:45.847642+0000","last_active":"2024-04-15T15:18:46.874458+0000","last_peered":"2024-04-15T15:18:46.874458+0000","last_clean":"2024-04-15T15:18:46.874458+0000","last_became_active":"2024-04-15T15:18:45.845790+0000","last_became_peered":"2024-04-15T15:18:45.845790+0000","last_unstale":"2024-04-15T15:18:46.874458+0000","last_undegraded":"2024-04-15T15:18:46.874458+0000","last_fullsized":"2024-04-15T15:18:46.874458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:24:25.450305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.863926+0000","last_change":"2024-04-15T15:18:44.857029+0000","last_active":"2024-04-15T15:18:47.863926+0000","last_peered":"2024-04-15T15:18:47.863926+0000","last_clean":"2024-04-15T15:18:47.863926+0000","last_became_active":"2024-04-15T15:18:44.855203+0000","last_became_peered":"2024-04-15T15:18:44.855203+0000","last_unstale":"2024-04-15T15:18:47.863926+0000","last_undegraded":"2024-04-15T15:18:47.863926+0000","last_fullsized":"2024-04-15T15:18:47.863926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:34.796362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818209+0000","last_change":"2024-04-15T15:18:45.856661+0000","last_active":"2024-04-15T15:18:46.818209+0000","last_peered":"2024-04-15T15:18:46.818209+0000","last_clean":"2024-04-15T15:18:46.818209+0000","last_became_active":"2024-04-15T15:18:45.856443+0000","last_became_peered":"2024-04-15T15:18:45.856443+0000","last_unstale":"2024-04-15T15:18:46.818209+0000","last_undegraded":"2024-04-15T15:18:46.818209+0000","last_fullsized":"2024-04-15T15:18:46.818209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.566254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822757+0000","last_change":"2024-04-15T15:18:44.866401+0000","last_active":"2024-04-15T15:18:46.822757+0000","last_peered":"2024-04-15T15:18:46.822757+0000","last_clean":"2024-04-15T15:18:46.822757+0000","last_became_active":"2024-04-15T15:18:44.866099+0000","last_became_peered":"2024-04-15T15:18:44.866099+0000","last_unstale":"2024-04-15T15:18:46.822757+0000","last_undegraded":"2024-04-15T15:18:46.822757+0000","last_fullsized":"2024-04-15T15:18:46.822757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:15:03.621969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820130+0000","last_change":"2024-04-15T15:18:45.854183+0000","last_active":"2024-04-15T15:18:46.820130+0000","last_peered":"2024-04-15T15:18:46.820130+0000","last_clean":"2024-04-15T15:18:46.820130+0000","last_became_active":"2024-04-15T15:18:45.853850+0000","last_became_peered":"2024-04-15T15:18:45.853850+0000","last_unstale":"2024-04-15T15:18:46.820130+0000","last_undegraded":"2024-04-15T15:18:46.820130+0000","last_fullsized":"2024-04-15T15:18:46.820130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:32:58.852361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.864715+0000","last_change":"2024-04-15T15:18:44.857499+0000","last_active":"2024-04-15T15:18:47.864715+0000","last_peered":"2024-04-15T15:18:47.864715+0000","last_clean":"2024-04-15T15:18:47.864715+0000","last_became_active":"2024-04-15T15:18:44.857140+0000","last_became_peered":"2024-04-15T15:18:44.857140+0000","last_unstale":"2024-04-15T15:18:47.864715+0000","last_undegraded":"2024-04-15T15:18:47.864715+0000","last_fullsized":"2024-04-15T15:18:47.864715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.826627+0000","last_change":"2024-04-15T15:18:45.860607+0000","last_active":"2024-04-15T15:18:46.826627+0000","last_peered":"2024-04-15T15:18:46.826627+0000","last_clean":"2024-04-15T15:18:46.826627+0000","last_became_active":"2024-04-15T15:18:45.858730+0000","last_became_peered":"2024-04-15T15:18:45.858730+0000","last_unstale":"2024-04-15T15:18:46.826627+0000","last_undegraded":"2024-04-15T15:18:46.826627+0000","last_fullsized":"2024-04-15T15:18:46.826627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:10:16.717594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870214+0000","last_change":"2024-04-15T15:18:44.858372+0000","last_active":"2024-04-15T15:18:46.870214+0000","last_peered":"2024-04-15T15:18:46.870214+0000","last_clean":"2024-04-15T15:18:46.870214+0000","last_became_active":"2024-04-15T15:18:44.858083+0000","last_became_peered":"2024-04-15T15:18:44.858083+0000","last_unstale":"2024-04-15T15:18:46.870214+0000","last_undegraded":"2024-04-15T15:18:46.870214+0000","last_fullsized":"2024-04-15T15:18:46.870214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824118+0000","last_change":"2024-04-15T15:18:45.864746+0000","last_active":"2024-04-15T15:18:46.824118+0000","last_peered":"2024-04-15T15:18:46.824118+0000","last_clean":"2024-04-15T15:18:46.824118+0000","last_became_active":"2024-04-15T15:18:45.858273+0000","last_became_peered":"2024-04-15T15:18:45.858273+0000","last_unstale":"2024-04-15T15:18:46.824118+0000","last_undegraded":"2024-04-15T15:18:46.824118+0000","last_fullsized":"2024-04-15T15:18:46.824118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:36.969295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.864820+0000","last_change":"2024-04-15T15:18:44.864232+0000","last_active":"2024-04-15T15:18:47.864820+0000","last_peered":"2024-04-15T15:18:47.864820+0000","last_clean":"2024-04-15T15:18:47.864820+0000","last_became_active":"2024-04-15T15:18:44.863837+0000","last_became_peered":"2024-04-15T15:18:44.863837+0000","last_unstale":"2024-04-15T15:18:47.864820+0000","last_undegraded":"2024-04-15T15:18:47.864820+0000","last_fullsized":"2024-04-15T15:18:47.864820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818747+0000","last_change":"2024-04-15T15:18:45.860687+0000","last_active":"2024-04-15T15:18:46.818747+0000","last_peered":"2024-04-15T15:18:46.818747+0000","last_clean":"2024-04-15T15:18:46.818747+0000","last_became_active":"2024-04-15T15:18:45.860227+0000","last_became_peered":"2024-04-15T15:18:45.860227+0000","last_unstale":"2024-04-15T15:18:46.818747+0000","last_undegraded":"2024-04-15T15:18:46.818747+0000","last_fullsized":"2024-04-15T15:18:46.818747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:29:14.132066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.817987+0000","last_change":"2024-04-15T15:18:41.162887+0000","last_active":"2024-04-15T15:18:46.817987+0000","last_peered":"2024-04-15T15:18:46.817987+0000","last_clean":"2024-04-15T15:18:46.817987+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:18:46.817987+0000","last_undegraded":"2024-04-15T15:18:46.817987+0000","last_fullsized":"2024-04-15T15:18:46.817987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_clean_scrub_stamp":"2024-04-15T15:18:39.644354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:32:00.047414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821321+0000","last_change":"2024-04-15T15:18:44.858206+0000","last_active":"2024-04-15T15:18:46.821321+0000","last_peered":"2024-04-15T15:18:46.821321+0000","last_clean":"2024-04-15T15:18:46.821321+0000","last_became_active":"2024-04-15T15:18:44.857694+0000","last_became_peered":"2024-04-15T15:18:44.857694+0000","last_unstale":"2024-04-15T15:18:46.821321+0000","last_undegraded":"2024-04-15T15:18:46.821321+0000","last_fullsized":"2024-04-15T15:18:46.821321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868291+0000","last_change":"2024-04-15T15:18:45.849224+0000","last_active":"2024-04-15T15:18:47.868291+0000","last_peered":"2024-04-15T15:18:47.868291+0000","last_clean":"2024-04-15T15:18:47.868291+0000","last_became_active":"2024-04-15T15:18:45.848961+0000","last_became_peered":"2024-04-15T15:18:45.848961+0000","last_unstale":"2024-04-15T15:18:47.868291+0000","last_undegraded":"2024-04-15T15:18:47.868291+0000","last_fullsized":"2024-04-15T15:18:47.868291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:00:19.716082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.817850+0000","last_change":"2024-04-15T15:18:44.859202+0000","last_active":"2024-04-15T15:18:46.817850+0000","last_peered":"2024-04-15T15:18:46.817850+0000","last_clean":"2024-04-15T15:18:46.817850+0000","last_became_active":"2024-04-15T15:18:44.858791+0000","last_became_peered":"2024-04-15T15:18:44.858791+0000","last_unstale":"2024-04-15T15:18:46.817850+0000","last_undegraded":"2024-04-15T15:18:46.817850+0000","last_fullsized":"2024-04-15T15:18:46.817850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.865928+0000","last_change":"2024-04-15T15:18:45.851981+0000","last_active":"2024-04-15T15:18:47.865928+0000","last_peered":"2024-04-15T15:18:47.865928+0000","last_clean":"2024-04-15T15:18:47.865928+0000","last_became_active":"2024-04-15T15:18:45.851653+0000","last_became_peered":"2024-04-15T15:18:45.851653+0000","last_unstale":"2024-04-15T15:18:47.865928+0000","last_undegraded":"2024-04-15T15:18:47.865928+0000","last_fullsized":"2024-04-15T15:18:47.865928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:16:52.507783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.864638+0000","last_change":"2024-04-15T15:18:44.849893+0000","last_active":"2024-04-15T15:18:47.864638+0000","last_peered":"2024-04-15T15:18:47.864638+0000","last_clean":"2024-04-15T15:18:47.864638+0000","last_became_active":"2024-04-15T15:18:44.848047+0000","last_became_peered":"2024-04-15T15:18:44.848047+0000","last_unstale":"2024-04-15T15:18:47.864638+0000","last_undegraded":"2024-04-15T15:18:47.864638+0000","last_fullsized":"2024-04-15T15:18:47.864638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869862+0000","last_change":"2024-04-15T15:18:45.844515+0000","last_active":"2024-04-15T15:18:46.869862+0000","last_peered":"2024-04-15T15:18:46.869862+0000","last_clean":"2024-04-15T15:18:46.869862+0000","last_became_active":"2024-04-15T15:18:45.844154+0000","last_became_peered":"2024-04-15T15:18:45.844154+0000","last_unstale":"2024-04-15T15:18:46.869862+0000","last_undegraded":"2024-04-15T15:18:46.869862+0000","last_fullsized":"2024-04-15T15:18:46.869862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823178+0000","last_change":"2024-04-15T15:18:44.862254+0000","last_active":"2024-04-15T15:18:46.823178+0000","last_peered":"2024-04-15T15:18:46.823178+0000","last_clean":"2024-04-15T15:18:46.823178+0000","last_became_active":"2024-04-15T15:18:44.861231+0000","last_became_peered":"2024-04-15T15:18:44.861231+0000","last_unstale":"2024-04-15T15:18:46.823178+0000","last_undegraded":"2024-04-15T15:18:46.823178+0000","last_fullsized":"2024-04-15T15:18:46.823178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.821451+0000","last_change":"2024-04-15T15:18:45.847966+0000","last_active":"2024-04-15T15:18:46.821451+0000","last_peered":"2024-04-15T15:18:46.821451+0000","last_clean":"2024-04-15T15:18:46.821451+0000","last_became_active":"2024-04-15T15:18:45.847628+0000","last_became_peered":"2024-04-15T15:18:45.847628+0000","last_unstale":"2024-04-15T15:18:46.821451+0000","last_undegraded":"2024-04-15T15:18:46.821451+0000","last_fullsized":"2024-04-15T15:18:46.821451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:35:55.040215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870108+0000","last_change":"2024-04-15T15:18:44.866681+0000","last_active":"2024-04-15T15:18:46.870108+0000","last_peered":"2024-04-15T15:18:46.870108+0000","last_clean":"2024-04-15T15:18:46.870108+0000","last_became_active":"2024-04-15T15:18:44.866366+0000","last_became_peered":"2024-04-15T15:18:44.866366+0000","last_unstale":"2024-04-15T15:18:46.870108+0000","last_undegraded":"2024-04-15T15:18:46.870108+0000","last_fullsized":"2024-04-15T15:18:46.870108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:27:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824652+0000","last_change":"2024-04-15T15:18:45.858129+0000","last_active":"2024-04-15T15:18:46.824652+0000","last_peered":"2024-04-15T15:18:46.824652+0000","last_clean":"2024-04-15T15:18:46.824652+0000","last_became_active":"2024-04-15T15:18:45.857824+0000","last_became_peered":"2024-04-15T15:18:45.857824+0000","last_unstale":"2024-04-15T15:18:46.824652+0000","last_undegraded":"2024-04-15T15:18:46.824652+0000","last_fullsized":"2024-04-15T15:18:46.824652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:36:03.614062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.820284+0000","last_change":"2024-04-15T15:18:44.866754+0000","last_active":"2024-04-15T15:18:46.820284+0000","last_peered":"2024-04-15T15:18:46.820284+0000","last_clean":"2024-04-15T15:18:46.820284+0000","last_became_active":"2024-04-15T15:18:44.866479+0000","last_became_peered":"2024-04-15T15:18:44.866479+0000","last_unstale":"2024-04-15T15:18:46.820284+0000","last_undegraded":"2024-04-15T15:18:46.820284+0000","last_fullsized":"2024-04-15T15:18:46.820284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:17:09.815187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.870141+0000","last_change":"2024-04-15T15:18:45.844997+0000","last_active":"2024-04-15T15:18:46.870141+0000","last_peered":"2024-04-15T15:18:46.870141+0000","last_clean":"2024-04-15T15:18:46.870141+0000","last_became_active":"2024-04-15T15:18:45.844589+0000","last_became_peered":"2024-04-15T15:18:45.844589+0000","last_unstale":"2024-04-15T15:18:46.870141+0000","last_undegraded":"2024-04-15T15:18:46.870141+0000","last_fullsized":"2024-04-15T15:18:46.870141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.873141+0000","last_change":"2024-04-15T15:18:44.861663+0000","last_active":"2024-04-15T15:18:46.873141+0000","last_peered":"2024-04-15T15:18:46.873141+0000","last_clean":"2024-04-15T15:18:46.873141+0000","last_became_active":"2024-04-15T15:18:44.861347+0000","last_became_peered":"2024-04-15T15:18:44.861347+0000","last_unstale":"2024-04-15T15:18:46.873141+0000","last_undegraded":"2024-04-15T15:18:46.873141+0000","last_fullsized":"2024-04-15T15:18:46.873141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:54.111084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868925+0000","last_change":"2024-04-15T15:18:45.854665+0000","last_active":"2024-04-15T15:18:47.868925+0000","last_peered":"2024-04-15T15:18:47.868925+0000","last_clean":"2024-04-15T15:18:47.868925+0000","last_became_active":"2024-04-15T15:18:45.853958+0000","last_became_peered":"2024-04-15T15:18:45.853958+0000","last_unstale":"2024-04-15T15:18:47.868925+0000","last_undegraded":"2024-04-15T15:18:47.868925+0000","last_fullsized":"2024-04-15T15:18:47.868925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.566254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.875345+0000","last_change":"2024-04-15T15:18:44.866078+0000","last_active":"2024-04-15T15:18:46.875345+0000","last_peered":"2024-04-15T15:18:46.875345+0000","last_clean":"2024-04-15T15:18:46.875345+0000","last_became_active":"2024-04-15T15:18:44.865730+0000","last_became_peered":"2024-04-15T15:18:44.865730+0000","last_unstale":"2024-04-15T15:18:46.875345+0000","last_undegraded":"2024-04-15T15:18:46.875345+0000","last_fullsized":"2024-04-15T15:18:46.875345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.869123+0000","last_change":"2024-04-15T15:18:45.841162+0000","last_active":"2024-04-15T15:18:47.869123+0000","last_peered":"2024-04-15T15:18:47.869123+0000","last_clean":"2024-04-15T15:18:47.869123+0000","last_became_active":"2024-04-15T15:18:45.840894+0000","last_became_peered":"2024-04-15T15:18:45.840894+0000","last_unstale":"2024-04-15T15:18:47.869123+0000","last_undegraded":"2024-04-15T15:18:47.869123+0000","last_fullsized":"2024-04-15T15:18:47.869123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:54:07.153141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819201+0000","last_change":"2024-04-15T15:18:44.862312+0000","last_active":"2024-04-15T15:18:46.819201+0000","last_peered":"2024-04-15T15:18:46.819201+0000","last_clean":"2024-04-15T15:18:46.819201+0000","last_became_active":"2024-04-15T15:18:44.862026+0000","last_became_peered":"2024-04-15T15:18:44.862026+0000","last_unstale":"2024-04-15T15:18:46.819201+0000","last_undegraded":"2024-04-15T15:18:46.819201+0000","last_fullsized":"2024-04-15T15:18:46.819201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:19:19.067311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.822643+0000","last_change":"2024-04-15T15:18:45.847316+0000","last_active":"2024-04-15T15:18:46.822643+0000","last_peered":"2024-04-15T15:18:46.822643+0000","last_clean":"2024-04-15T15:18:46.822643+0000","last_became_active":"2024-04-15T15:18:45.846987+0000","last_became_peered":"2024-04-15T15:18:45.846987+0000","last_unstale":"2024-04-15T15:18:46.822643+0000","last_undegraded":"2024-04-15T15:18:46.822643+0000","last_fullsized":"2024-04-15T15:18:46.822643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:58:24.586889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869976+0000","last_change":"2024-04-15T15:18:44.859986+0000","last_active":"2024-04-15T15:18:46.869976+0000","last_peered":"2024-04-15T15:18:46.869976+0000","last_clean":"2024-04-15T15:18:46.869976+0000","last_became_active":"2024-04-15T15:18:44.859733+0000","last_became_peered":"2024-04-15T15:18:44.859733+0000","last_unstale":"2024-04-15T15:18:46.869976+0000","last_undegraded":"2024-04-15T15:18:46.869976+0000","last_fullsized":"2024-04-15T15:18:46.869976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:26.555787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.865523+0000","last_change":"2024-04-15T15:18:45.846102+0000","last_active":"2024-04-15T15:18:47.865523+0000","last_peered":"2024-04-15T15:18:47.865523+0000","last_clean":"2024-04-15T15:18:47.865523+0000","last_became_active":"2024-04-15T15:18:45.845715+0000","last_became_peered":"2024-04-15T15:18:45.845715+0000","last_unstale":"2024-04-15T15:18:47.865523+0000","last_undegraded":"2024-04-15T15:18:47.865523+0000","last_fullsized":"2024-04-15T15:18:47.865523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:27.111247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.869992+0000","last_change":"2024-04-15T15:18:44.862018+0000","last_active":"2024-04-15T15:18:46.869992+0000","last_peered":"2024-04-15T15:18:46.869992+0000","last_clean":"2024-04-15T15:18:46.869992+0000","last_became_active":"2024-04-15T15:18:44.861663+0000","last_became_peered":"2024-04-15T15:18:44.861663+0000","last_unstale":"2024-04-15T15:18:46.869992+0000","last_undegraded":"2024-04-15T15:18:46.869992+0000","last_fullsized":"2024-04-15T15:18:46.869992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:34:16.947015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868976+0000","last_change":"2024-04-15T15:18:45.852079+0000","last_active":"2024-04-15T15:18:47.868976+0000","last_peered":"2024-04-15T15:18:47.868976+0000","last_clean":"2024-04-15T15:18:47.868976+0000","last_became_active":"2024-04-15T15:18:45.851281+0000","last_became_peered":"2024-04-15T15:18:45.851281+0000","last_unstale":"2024-04-15T15:18:47.868976+0000","last_undegraded":"2024-04-15T15:18:47.868976+0000","last_fullsized":"2024-04-15T15:18:47.868976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:53.984837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.819899+0000","last_change":"2024-04-15T15:18:44.867447+0000","last_active":"2024-04-15T15:18:46.819899+0000","last_peered":"2024-04-15T15:18:46.819899+0000","last_clean":"2024-04-15T15:18:46.819899+0000","last_became_active":"2024-04-15T15:18:44.867171+0000","last_became_peered":"2024-04-15T15:18:44.867171+0000","last_unstale":"2024-04-15T15:18:46.819899+0000","last_undegraded":"2024-04-15T15:18:46.819899+0000","last_fullsized":"2024-04-15T15:18:46.819899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:01:26.555787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.874593+0000","last_change":"2024-04-15T15:18:45.861329+0000","last_active":"2024-04-15T15:18:46.874593+0000","last_peered":"2024-04-15T15:18:46.874593+0000","last_clean":"2024-04-15T15:18:46.874593+0000","last_became_active":"2024-04-15T15:18:45.859564+0000","last_became_peered":"2024-04-15T15:18:45.859564+0000","last_unstale":"2024-04-15T15:18:46.874593+0000","last_undegraded":"2024-04-15T15:18:46.874593+0000","last_fullsized":"2024-04-15T15:18:46.874593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:09:18.162344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.824729+0000","last_change":"2024-04-15T15:18:45.831940+0000","last_active":"2024-04-15T15:18:46.824729+0000","last_peered":"2024-04-15T15:18:46.824729+0000","last_clean":"2024-04-15T15:18:46.824729+0000","last_became_active":"2024-04-15T15:18:45.829915+0000","last_became_peered":"2024-04-15T15:18:45.829915+0000","last_unstale":"2024-04-15T15:18:46.824729+0000","last_undegraded":"2024-04-15T15:18:46.824729+0000","last_fullsized":"2024-04-15T15:18:46.824729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:06:25.954239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818726+0000","last_change":"2024-04-15T15:18:44.869358+0000","last_active":"2024-04-15T15:18:46.818726+0000","last_peered":"2024-04-15T15:18:46.818726+0000","last_clean":"2024-04-15T15:18:46.818726+0000","last_became_active":"2024-04-15T15:18:44.869040+0000","last_became_peered":"2024-04-15T15:18:44.869040+0000","last_unstale":"2024-04-15T15:18:46.818726+0000","last_undegraded":"2024-04-15T15:18:46.818726+0000","last_fullsized":"2024-04-15T15:18:46.818726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:54.111084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.823654+0000","last_change":"2024-04-15T15:18:45.851111+0000","last_active":"2024-04-15T15:18:46.823654+0000","last_peered":"2024-04-15T15:18:46.823654+0000","last_clean":"2024-04-15T15:18:46.823654+0000","last_became_active":"2024-04-15T15:18:45.850824+0000","last_became_peered":"2024-04-15T15:18:45.850824+0000","last_unstale":"2024-04-15T15:18:46.823654+0000","last_undegraded":"2024-04-15T15:18:46.823654+0000","last_fullsized":"2024-04-15T15:18:46.823654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:53.984837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:46.818964+0000","last_change":"2024-04-15T15:18:44.864267+0000","last_active":"2024-04-15T15:18:46.818964+0000","last_peered":"2024-04-15T15:18:46.818964+0000","last_clean":"2024-04-15T15:18:46.818964+0000","last_became_active":"2024-04-15T15:18:44.863962+0000","last_became_peered":"2024-04-15T15:18:44.863962+0000","last_unstale":"2024-04-15T15:18:46.818964+0000","last_undegraded":"2024-04-15T15:18:46.818964+0000","last_fullsized":"2024-04-15T15:18:46.818964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:22:53.739928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868055+0000","last_change":"2024-04-15T15:18:45.845359+0000","last_active":"2024-04-15T15:18:47.868055+0000","last_peered":"2024-04-15T15:18:47.868055+0000","last_clean":"2024-04-15T15:18:47.868055+0000","last_became_active":"2024-04-15T15:18:45.845087+0000","last_became_peered":"2024-04-15T15:18:45.845087+0000","last_unstale":"2024-04-15T15:18:47.868055+0000","last_undegraded":"2024-04-15T15:18:47.868055+0000","last_fullsized":"2024-04-15T15:18:47.868055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:12:18.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868578+0000","last_change":"2024-04-15T15:18:44.864497+0000","last_active":"2024-04-15T15:18:47.868578+0000","last_peered":"2024-04-15T15:18:47.868578+0000","last_clean":"2024-04-15T15:18:47.868578+0000","last_became_active":"2024-04-15T15:18:44.864122+0000","last_became_peered":"2024-04-15T15:18:44.864122+0000","last_unstale":"2024-04-15T15:18:47.868578+0000","last_undegraded":"2024-04-15T15:18:47.868578+0000","last_fullsized":"2024-04-15T15:18:47.868578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:30:50.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868175+0000","last_change":"2024-04-15T15:18:45.861978+0000","last_active":"2024-04-15T15:18:47.868175+0000","last_peered":"2024-04-15T15:18:47.868175+0000","last_clean":"2024-04-15T15:18:47.868175+0000","last_became_active":"2024-04-15T15:18:45.861597+0000","last_became_peered":"2024-04-15T15:18:45.861597+0000","last_unstale":"2024-04-15T15:18:47.868175+0000","last_undegraded":"2024-04-15T15:18:47.868175+0000","last_fullsized":"2024-04-15T15:18:47.868175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:44.816947+0000","last_clean_scrub_stamp":"2024-04-15T15:18:44.816947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:20:02.921227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:18:47.868895+0000","last_change":"2024-04-15T15:18:44.864325+0000","last_active":"2024-04-15T15:18:47.868895+0000","last_peered":"2024-04-15T15:18:47.868895+0000","last_clean":"2024-04-15T15:18:47.868895+0000","last_became_active":"2024-04-15T15:18:44.863985+0000","last_became_peered":"2024-04-15T15:18:44.863985+0000","last_unstale":"2024-04-15T15:18:47.868895+0000","last_undegraded":"2024-04-15T15:18:47.868895+0000","last_fullsized":"2024-04-15T15:18:47.868895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:43.681752+0000","last_clean_scrub_stamp":"2024-04-15T15:18:43.681752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:56:01.546965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":823296,"data_stored":669858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":79034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":237568,"data_stored":80520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":78908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":79068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":831488,"data_stored":669914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:18:56.695 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-15T15:18:56.695 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-15T15:18:56.695 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-15T15:18:56.695 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-15T15:18:56.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.955+0000 7fb928e00640 1 Processor -- start 2024-04-15T15:18:56.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.956+0000 7fb928e00640 1 -- start start 2024-04-15T15:18:56.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.957+0000 7fb928e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9241481f0 0x7fb92415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.957+0000 7fb928e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb9240affc0 0x7fb9240721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.957+0000 7fb928e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9241485d0 con 0x7fb9240b5560 2024-04-15T15:18:56.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.957+0000 7fb928e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fb92415ffc0 con 0x7fb9240affc0 2024-04-15T15:18:56.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.957+0000 7fb928e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb9240aef20 con 0x7fb9241481f0 2024-04-15T15:18:56.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.958+0000 7fb921e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9241481f0 0x7fb92415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.958+0000 7fb921e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9241481f0 0x7fb92415fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:43260/0 (socket says 172.21.15.49:43260) 2024-04-15T15:18:56.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.958+0000 7fb921e00640 1 -- 172.21.15.49:0/2339979522 learned_addr learned my addr 172.21.15.49:0/2339979522 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:56.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.958+0000 7fb919400640 1 --2- 172.21.15.49:0/2339979522 >> v2:172.21.15.107:3300/0 conn(0x7fb9240affc0 0x7fb9240721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.959+0000 7fb921e00640 1 -- 172.21.15.49:0/2339979522 >> v2:172.21.15.107:3300/0 conn(0x7fb9240affc0 msgr2=0x7fb9240721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:56.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.959+0000 7fb921e00640 1 --2- 172.21.15.49:0/2339979522 >> v2:172.21.15.107:3300/0 conn(0x7fb9240affc0 0x7fb9240721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.959+0000 7fb921e00640 1 -- 172.21.15.49:0/2339979522 >> v1:172.21.15.107:6789/0 conn(0x7fb9240b5560 legacy=0x7fb9240b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.959+0000 7fb921e00640 1 -- 172.21.15.49:0/2339979522 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9240af200 con 0x7fb9241481f0 2024-04-15T15:18:56.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.960+0000 7fb921e00640 1 --2- 172.21.15.49:0/2339979522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9241481f0 0x7fb92415fa80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb914009870 tx=0x7fb91402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=421f92994528863 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:56.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.960+0000 7fb921400640 1 -- 172.21.15.49:0/2339979522 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914034070 con 0x7fb9241481f0 2024-04-15T15:18:56.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.960+0000 7fb921400640 1 -- 172.21.15.49:0/2339979522 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb914030840 con 0x7fb9241481f0 2024-04-15T15:18:56.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.960+0000 7fb921400640 1 -- 172.21.15.49:0/2339979522 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914030c10 con 0x7fb9241481f0 2024-04-15T15:18:56.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.961+0000 7fb928e00640 1 -- 172.21.15.49:0/2339979522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9241481f0 msgr2=0x7fb92415fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.961+0000 7fb928e00640 1 --2- 172.21.15.49:0/2339979522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9241481f0 0x7fb92415fa80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb914009870 tx=0x7fb91402b980 comp rx=0 tx=0).stop 2024-04-15T15:18:56.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.961+0000 7fb928e00640 1 -- 172.21.15.49:0/2339979522 shutdown_connections 2024-04-15T15:18:56.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.961+0000 7fb928e00640 1 --2- 172.21.15.49:0/2339979522 >> v2:172.21.15.107:3300/0 conn(0x7fb9240affc0 0x7fb9240721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.961+0000 7fb928e00640 1 --2- 172.21.15.49:0/2339979522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9241481f0 0x7fb92415fa80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.961+0000 7fb928e00640 1 -- 172.21.15.49:0/2339979522 >> 172.21.15.49:0/2339979522 conn(0x7fb9240b27b0 msgr2=0x7fb9240b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:56.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.961+0000 7fb928e00640 1 -- 172.21.15.49:0/2339979522 shutdown_connections 2024-04-15T15:18:56.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.961+0000 7fb928e00640 1 -- 172.21.15.49:0/2339979522 wait complete. 2024-04-15T15:18:56.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.963+0000 7fb928e00640 1 Processor -- start 2024-04-15T15:18:56.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.963+0000 7fb928e00640 1 -- start start 2024-04-15T15:18:56.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.964+0000 7fb928e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9240affc0 0x7fb924138ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.964+0000 7fb928e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240b5560 0x7fb924139840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.964+0000 7fb928e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9241481f0 0x7fb924139d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.964+0000 7fb928e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240afaa0 con 0x7fb9241481f0 2024-04-15T15:18:56.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.964+0000 7fb928e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240af7c0 con 0x7fb9240b5560 2024-04-15T15:18:56.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.964+0000 7fb928e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb924072720 con 0x7fb9240affc0 2024-04-15T15:18:56.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.964+0000 7fb922800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9241481f0 0x7fb924139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.964+0000 7fb922800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9241481f0 0x7fb924139d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39918/0 (socket says 172.21.15.49:39918) 2024-04-15T15:18:56.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.964+0000 7fb922800640 1 -- 172.21.15.49:0/2350499502 learned_addr learned my addr 172.21.15.49:0/2350499502 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:56.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb921e00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9240affc0 0x7fb924138ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb919400640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240b5560 0x7fb924139840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb922800640 1 -- 172.21.15.49:0/2350499502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9240affc0 msgr2=0x7fb924138ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb922800640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9240affc0 0x7fb924138ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb922800640 1 -- 172.21.15.49:0/2350499502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240b5560 msgr2=0x7fb924139840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:56.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb922800640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240b5560 0x7fb924139840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:56.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb922800640 1 -- 172.21.15.49:0/2350499502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb924139410 con 0x7fb9241481f0 2024-04-15T15:18:56.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb921e00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9240affc0 0x7fb924138ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:56.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb919400640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240b5560 0x7fb924139840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:56.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb922800640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9241481f0 0x7fb924139d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00c760 tx=0x7fb91c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:56.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb91aa00640 1 -- 172.21.15.49:0/2350499502 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c003b90 con 0x7fb9241481f0 2024-04-15T15:18:56.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb91aa00640 1 -- 172.21.15.49:0/2350499502 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c003d30 con 0x7fb9241481f0 2024-04-15T15:18:56.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb91aa00640 1 -- 172.21.15.49:0/2350499502 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00aa90 con 0x7fb9241481f0 2024-04-15T15:18:56.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb928e00640 1 -- 172.21.15.49:0/2350499502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9240cdb20 con 0x7fb9241481f0 2024-04-15T15:18:56.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.965+0000 7fb928e00640 1 -- 172.21.15.49:0/2350499502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9240ce0e0 con 0x7fb9241481f0 2024-04-15T15:18:56.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.966+0000 7fb928e00640 1 -- 172.21.15.49:0/2350499502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb924068c80 con 0x7fb9241481f0 2024-04-15T15:18:56.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.967+0000 7fb91aa00640 1 -- 172.21.15.49:0/2350499502 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb91c00ac30 con 0x7fb9241481f0 2024-04-15T15:18:56.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.968+0000 7fb91aa00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb90006c8f0 0x7fb90006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:56.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.968+0000 7fb91aa00640 1 -- 172.21.15.49:0/2350499502 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb91c08ba90 con 0x7fb9241481f0 2024-04-15T15:18:56.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.969+0000 7fb921e00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb90006c8f0 0x7fb90006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:56.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.969+0000 7fb921e00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb90006c8f0 0x7fb90006edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb914036000 tx=0x7fb914008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:56.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:56.973+0000 7fb91aa00640 1 -- 172.21.15.49:0/2350499502 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb91c059e20 con 0x7fb9241481f0 2024-04-15T15:18:57.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.206+0000 7fb928e00640 1 -- 172.21.15.49:0/2350499502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb924145d60 con 0x7fb9241481f0 2024-04-15T15:18:57.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.207+0000 7fb91aa00640 1 -- 172.21.15.49:0/2350499502 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb91c059720 con 0x7fb9241481f0 2024-04-15T15:18:57.208 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:57.208 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 -- 172.21.15.49:0/2350499502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb90006c8f0 msgr2=0x7fb90006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb90006c8f0 0x7fb90006edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb914036000 tx=0x7fb914008030 comp rx=0 tx=0).stop 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 -- 172.21.15.49:0/2350499502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9241481f0 msgr2=0x7fb924139d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9241481f0 0x7fb924139d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00c760 tx=0x7fb91c00cc20 comp rx=0 tx=0).stop 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 -- 172.21.15.49:0/2350499502 shutdown_connections 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb90006c8f0 0x7fb90006edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9241481f0 0x7fb924139d80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240b5560 0x7fb924139840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 --2- 172.21.15.49:0/2350499502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9240affc0 0x7fb924138ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 -- 172.21.15.49:0/2350499502 >> 172.21.15.49:0/2350499502 conn(0x7fb9240b27b0 msgr2=0x7fb924145650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:57.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 -- 172.21.15.49:0/2350499502 shutdown_connections 2024-04-15T15:18:57.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.209+0000 7fb928e00640 1 -- 172.21.15.49:0/2350499502 wait complete. 2024-04-15T15:18:57.231 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-15T15:18:57.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:57.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.454+0000 7f4437400640 1 Processor -- start 2024-04-15T15:18:57.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.454+0000 7f4437400640 1 -- start start 2024-04-15T15:18:57.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.456+0000 7f4437400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44301461f0 0x7f44300ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.456+0000 7f4437400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f44300cedf0 0x7f44300b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.456+0000 7f4437400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44300b4be0 con 0x7f44300b3e10 2024-04-15T15:18:57.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.456+0000 7f4437400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f44300a9640 con 0x7f44301461f0 2024-04-15T15:18:57.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.456+0000 7f4437400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f44300cf250 con 0x7f44300cedf0 2024-04-15T15:18:57.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.457+0000 7f4434a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44301461f0 0x7f44300ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.457+0000 7f4434a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44301461f0 0x7f44300ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39920/0 (socket says 172.21.15.49:39920) 2024-04-15T15:18:57.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.457+0000 7f4434a00640 1 -- 172.21.15.49:0/2359761344 learned_addr learned my addr 172.21.15.49:0/2359761344 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:57.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.457+0000 7f442fe00640 1 --2- 172.21.15.49:0/2359761344 >> v2:172.21.15.107:3300/0 conn(0x7f44300cedf0 0x7f44300b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.457+0000 7f4434a00640 1 -- 172.21.15.49:0/2359761344 >> v2:172.21.15.107:3300/0 conn(0x7f44300cedf0 msgr2=0x7f44300b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:57.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.457+0000 7f4434a00640 1 --2- 172.21.15.49:0/2359761344 >> v2:172.21.15.107:3300/0 conn(0x7f44300cedf0 0x7f44300b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.457+0000 7f4434a00640 1 -- 172.21.15.49:0/2359761344 >> v1:172.21.15.107:6789/0 conn(0x7f44300b3e10 legacy=0x7f44300cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:57.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.457+0000 7f4434a00640 1 -- 172.21.15.49:0/2359761344 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44300b4620 con 0x7f44301461f0 2024-04-15T15:18:57.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.457+0000 7f442fe00640 1 --2- 172.21.15.49:0/2359761344 >> v2:172.21.15.107:3300/0 conn(0x7f44300cedf0 0x7f44300b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:57.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.458+0000 7f4434a00640 1 --2- 172.21.15.49:0/2359761344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44301461f0 0x7f44300ce8b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4430073560 tx=0x7f441c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0c7a1d5c18c687 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:57.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f442f400640 1 -- 172.21.15.49:0/2359761344 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c035020 con 0x7f44301461f0 2024-04-15T15:18:57.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f442f400640 1 -- 172.21.15.49:0/2359761344 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f441c031870 con 0x7f44301461f0 2024-04-15T15:18:57.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f442f400640 1 -- 172.21.15.49:0/2359761344 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c031c40 con 0x7f44301461f0 2024-04-15T15:18:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f4437400640 1 -- 172.21.15.49:0/2359761344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44301461f0 msgr2=0x7f44300ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f4437400640 1 --2- 172.21.15.49:0/2359761344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44301461f0 0x7f44300ce8b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4430073560 tx=0x7f441c02b910 comp rx=0 tx=0).stop 2024-04-15T15:18:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f4437400640 1 -- 172.21.15.49:0/2359761344 shutdown_connections 2024-04-15T15:18:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f4437400640 1 --2- 172.21.15.49:0/2359761344 >> v2:172.21.15.107:3300/0 conn(0x7f44300cedf0 0x7f44300b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f4437400640 1 --2- 172.21.15.49:0/2359761344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44301461f0 0x7f44300ce8b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f4437400640 1 -- 172.21.15.49:0/2359761344 >> 172.21.15.49:0/2359761344 conn(0x7f44300b27b0 msgr2=0x7f44300b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:57.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f4437400640 1 -- 172.21.15.49:0/2359761344 shutdown_connections 2024-04-15T15:18:57.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.459+0000 7f4437400640 1 -- 172.21.15.49:0/2359761344 wait complete. 2024-04-15T15:18:57.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.461+0000 7f4437400640 1 Processor -- start 2024-04-15T15:18:57.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.461+0000 7f4437400640 1 -- start start 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.462+0000 7f4437400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44300b3e10 0x7f44301f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.462+0000 7f4437400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44300cedf0 0x7f44301e7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.462+0000 7f4437400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44301461f0 0x7f44301e7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.462+0000 7f4437400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f443015d290 con 0x7f44300cedf0 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.462+0000 7f4437400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f443015cfb0 con 0x7f44301461f0 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.462+0000 7f4437400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f44301465d0 con 0x7f44300b3e10 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f442fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44300cedf0 0x7f44301e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f442fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44300cedf0 0x7f44301e7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39922/0 (socket says 172.21.15.49:39922) 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f442fe00640 1 -- 172.21.15.49:0/172768103 learned_addr learned my addr 172.21.15.49:0/172768103 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f4435400640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44301461f0 0x7f44301e7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f4434a00640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44300b3e10 0x7f44301f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f442fe00640 1 -- 172.21.15.49:0/172768103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44300b3e10 msgr2=0x7f44301f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f442fe00640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44300b3e10 0x7f44301f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f442fe00640 1 -- 172.21.15.49:0/172768103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44301461f0 msgr2=0x7f44301e7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f442fe00640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44301461f0 0x7f44301e7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f442fe00640 1 -- 172.21.15.49:0/172768103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44300cd1c0 con 0x7f44300cedf0 2024-04-15T15:18:57.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.463+0000 7f4434a00640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44300b3e10 0x7f44301f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:57.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.464+0000 7f442fe00640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44300cedf0 0x7f44301e7990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f442000c7f0 tx=0x7f442000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:57.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.464+0000 7f442cc00640 1 -- 172.21.15.49:0/172768103 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420015070 con 0x7f44300cedf0 2024-04-15T15:18:57.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.464+0000 7f442cc00640 1 -- 172.21.15.49:0/172768103 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420007a30 con 0x7f44300cedf0 2024-04-15T15:18:57.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.464+0000 7f442cc00640 1 -- 172.21.15.49:0/172768103 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420007e00 con 0x7f44300cedf0 2024-04-15T15:18:57.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.464+0000 7f4437400640 1 -- 172.21.15.49:0/172768103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44300ccee0 con 0x7f44300cedf0 2024-04-15T15:18:57.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.464+0000 7f4437400640 1 -- 172.21.15.49:0/172768103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44301e53d0 con 0x7f44300cedf0 2024-04-15T15:18:57.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.465+0000 7f4437400640 1 -- 172.21.15.49:0/172768103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4430068c80 con 0x7f44300cedf0 2024-04-15T15:18:57.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.466+0000 7f442cc00640 1 -- 172.21.15.49:0/172768103 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4420018070 con 0x7f44300cedf0 2024-04-15T15:18:57.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.467+0000 7f442cc00640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f441006c8f0 0x7f441006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.467+0000 7f442cc00640 1 -- 172.21.15.49:0/172768103 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f442008bb10 con 0x7f44300cedf0 2024-04-15T15:18:57.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.467+0000 7f4434a00640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f441006c8f0 0x7f441006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.468+0000 7f4434a00640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f441006c8f0 0x7f441006edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f441c036000 tx=0x7f441c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.471+0000 7f442cc00640 1 -- 172.21.15.49:0/172768103 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4420059f40 con 0x7f44300cedf0 2024-04-15T15:18:57.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.678+0000 7f4437400640 1 -- 172.21.15.49:0/172768103 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4430141b30 con 0x7f44300cedf0 2024-04-15T15:18:57.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.681+0000 7f442cc00640 1 -- 172.21.15.49:0/172768103 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7f4420059840 con 0x7f44300cedf0 2024-04-15T15:18:57.683 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:57.683 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:18:57.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 -- 172.21.15.49:0/172768103 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f441006c8f0 msgr2=0x7f441006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f441006c8f0 0x7f441006edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f441c036000 tx=0x7f441c008030 comp rx=0 tx=0).stop 2024-04-15T15:18:57.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 -- 172.21.15.49:0/172768103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44300cedf0 msgr2=0x7f44301e7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44300cedf0 0x7f44301e7990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f442000c7f0 tx=0x7f442000cbf0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 -- 172.21.15.49:0/172768103 shutdown_connections 2024-04-15T15:18:57.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f441006c8f0 0x7f441006edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44301461f0 0x7f44301e7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44300cedf0 0x7f44301e7990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 --2- 172.21.15.49:0/172768103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44300b3e10 0x7f44301f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 -- 172.21.15.49:0/172768103 >> 172.21.15.49:0/172768103 conn(0x7f44300b27b0 msgr2=0x7f4430144060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 -- 172.21.15.49:0/172768103 shutdown_connections 2024-04-15T15:18:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.684+0000 7f4437400640 1 -- 172.21.15.49:0/172768103 wait complete. 2024-04-15T15:18:57.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:18:57.704 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:18:57.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.927+0000 7f0140a00640 1 Processor -- start 2024-04-15T15:18:57.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.928+0000 7f0140a00640 1 -- start start 2024-04-15T15:18:57.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.929+0000 7f0140a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f013c0d4e30 0x7f013c0d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.929+0000 7f0140a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f013c0cede0 0x7f013c0cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.929+0000 7f0140a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f013c15ec60 con 0x7f013c15e880 2024-04-15T15:18:57.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.929+0000 7f0140a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f013c0cfaf0 con 0x7f013c0d4e30 2024-04-15T15:18:57.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.929+0000 7f0140a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f013c0d2930 con 0x7f013c0cede0 2024-04-15T15:18:57.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.930+0000 7f0139e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f013c0d4e30 0x7f013c0d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.930+0000 7f0139e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f013c0d4e30 0x7f013c0d3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39928/0 (socket says 172.21.15.49:39928) 2024-04-15T15:18:57.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.930+0000 7f0139e00640 1 -- 172.21.15.49:0/4059779127 learned_addr learned my addr 172.21.15.49:0/4059779127 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:57.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.930+0000 7f013b200640 1 --2- 172.21.15.49:0/4059779127 >> v2:172.21.15.107:3300/0 conn(0x7f013c0cede0 0x7f013c0cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.930+0000 7f0139e00640 1 -- 172.21.15.49:0/4059779127 >> v2:172.21.15.107:3300/0 conn(0x7f013c0cede0 msgr2=0x7f013c0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:57.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.930+0000 7f0139e00640 1 --2- 172.21.15.49:0/4059779127 >> v2:172.21.15.107:3300/0 conn(0x7f013c0cede0 0x7f013c0cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.930+0000 7f0139e00640 1 -- 172.21.15.49:0/4059779127 >> v1:172.21.15.107:6789/0 conn(0x7f013c15e880 legacy=0x7f013c0d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.930+0000 7f0139e00640 1 -- 172.21.15.49:0/4059779127 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c0d2c10 con 0x7f013c0d4e30 2024-04-15T15:18:57.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.931+0000 7f0139e00640 1 --2- 172.21.15.49:0/4059779127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0d4e30 0x7f013c0d3410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f013c0b7670 tx=0x7f012c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe44089fd0183533 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:57.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.932+0000 7f0139400640 1 -- 172.21.15.49:0/4059779127 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c02cbb0 con 0x7f013c0d4e30 2024-04-15T15:18:57.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.932+0000 7f0139400640 1 -- 172.21.15.49:0/4059779127 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f012c02cd50 con 0x7f013c0d4e30 2024-04-15T15:18:57.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.932+0000 7f0139400640 1 -- 172.21.15.49:0/4059779127 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c034610 con 0x7f013c0d4e30 2024-04-15T15:18:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.933+0000 7f0140a00640 1 -- 172.21.15.49:0/4059779127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0d4e30 msgr2=0x7f013c0d3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.933+0000 7f0140a00640 1 --2- 172.21.15.49:0/4059779127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0d4e30 0x7f013c0d3410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f013c0b7670 tx=0x7f012c02c010 comp rx=0 tx=0).stop 2024-04-15T15:18:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.933+0000 7f0140a00640 1 -- 172.21.15.49:0/4059779127 shutdown_connections 2024-04-15T15:18:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.933+0000 7f0140a00640 1 --2- 172.21.15.49:0/4059779127 >> v2:172.21.15.107:3300/0 conn(0x7f013c0cede0 0x7f013c0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.933+0000 7f0140a00640 1 --2- 172.21.15.49:0/4059779127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0d4e30 0x7f013c0d3410 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.933+0000 7f0140a00640 1 -- 172.21.15.49:0/4059779127 >> 172.21.15.49:0/4059779127 conn(0x7f013c073480 msgr2=0x7f013c073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.933+0000 7f0140a00640 1 -- 172.21.15.49:0/4059779127 shutdown_connections 2024-04-15T15:18:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.933+0000 7f0140a00640 1 -- 172.21.15.49:0/4059779127 wait complete. 2024-04-15T15:18:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.934+0000 7f0140a00640 1 Processor -- start 2024-04-15T15:18:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.935+0000 7f0140a00640 1 -- start start 2024-04-15T15:18:57.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f0140a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0cede0 0x7f013c0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f0140a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c0d4e30 0x7f013c0b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f0140a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f013c15e880 0x7f013c14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f0140a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f013c0b7790 con 0x7f013c0cede0 2024-04-15T15:18:57.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f0140a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f013c0cf790 con 0x7f013c0d4e30 2024-04-15T15:18:57.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f0140a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f013c0b7910 con 0x7f013c15e880 2024-04-15T15:18:57.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f013a800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0cede0 0x7f013c0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f013a800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0cede0 0x7f013c0cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39940/0 (socket says 172.21.15.49:39940) 2024-04-15T15:18:57.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f013a800640 1 -- 172.21.15.49:0/542589979 learned_addr learned my addr 172.21.15.49:0/542589979 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:57.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f0139e00640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c0d4e30 0x7f013c0b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f013b200640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f013c15e880 0x7f013c14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f013a800640 1 -- 172.21.15.49:0/542589979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f013c15e880 msgr2=0x7f013c14f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f013a800640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f013c15e880 0x7f013c14f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f013a800640 1 -- 172.21.15.49:0/542589979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c0d4e30 msgr2=0x7f013c0b8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:57.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f013a800640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c0d4e30 0x7f013c0b8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:57.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f013a800640 1 -- 172.21.15.49:0/542589979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c159b40 con 0x7f013c0cede0 2024-04-15T15:18:57.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.936+0000 7f013b200640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f013c15e880 0x7f013c14f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:57.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.937+0000 7f013a800640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0cede0 0x7f013c0cae20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f012400c400 tx=0x7f012400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:57.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.937+0000 7f0132a00640 1 -- 172.21.15.49:0/542589979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124017070 con 0x7f013c0cede0 2024-04-15T15:18:57.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.937+0000 7f0132a00640 1 -- 172.21.15.49:0/542589979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01240052d0 con 0x7f013c0cede0 2024-04-15T15:18:57.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.937+0000 7f0132a00640 1 -- 172.21.15.49:0/542589979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124003830 con 0x7f013c0cede0 2024-04-15T15:18:57.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.937+0000 7f0140a00640 1 -- 172.21.15.49:0/542589979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c159e20 con 0x7f013c0cede0 2024-04-15T15:18:57.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.937+0000 7f0140a00640 1 -- 172.21.15.49:0/542589979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c15a3e0 con 0x7f013c0cede0 2024-04-15T15:18:57.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.938+0000 7f0140a00640 1 -- 172.21.15.49:0/542589979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013c0d55a0 con 0x7f013c0cede0 2024-04-15T15:18:57.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.939+0000 7f0132a00640 1 -- 172.21.15.49:0/542589979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f012401a070 con 0x7f013c0cede0 2024-04-15T15:18:57.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.940+0000 7f0132a00640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f011806c8f0 0x7f011806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:57.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.940+0000 7f0132a00640 1 -- 172.21.15.49:0/542589979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f012408af70 con 0x7f013c0cede0 2024-04-15T15:18:57.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.940+0000 7f0139e00640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f011806c8f0 0x7f011806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:57.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.941+0000 7f0139e00640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f011806c8f0 0x7f011806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f012c035000 tx=0x7f012c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:57.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:57.945+0000 7f0132a00640 1 -- 172.21.15.49:0/542589979 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f013c0d55a0 con 0x7f013c0cede0 2024-04-15T15:18:58.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.147+0000 7f0140a00640 1 -- 172.21.15.49:0/542589979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f013c161ce0 con 0x7f013c0cede0 2024-04-15T15:18:58.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.148+0000 7f0132a00640 1 -- 172.21.15.49:0/542589979 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f0124059300 con 0x7f013c0cede0 2024-04-15T15:18:58.149 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:18:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.150+0000 7f0140a00640 1 -- 172.21.15.49:0/542589979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f011806c8f0 msgr2=0x7f011806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.150+0000 7f0140a00640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f011806c8f0 0x7f011806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f012c035000 tx=0x7f012c030000 comp rx=0 tx=0).stop 2024-04-15T15:18:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.150+0000 7f0140a00640 1 -- 172.21.15.49:0/542589979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0cede0 msgr2=0x7f013c0cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.150+0000 7f0140a00640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0cede0 0x7f013c0cae20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f012400c400 tx=0x7f012400c8c0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.150+0000 7f0140a00640 1 -- 172.21.15.49:0/542589979 shutdown_connections 2024-04-15T15:18:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.150+0000 7f0140a00640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f011806c8f0 0x7f011806edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.150+0000 7f0140a00640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f013c15e880 0x7f013c14f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.150+0000 7f0140a00640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c0d4e30 0x7f013c0b8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.150+0000 7f0140a00640 1 --2- 172.21.15.49:0/542589979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f013c0cede0 0x7f013c0cae20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.150+0000 7f0140a00640 1 -- 172.21.15.49:0/542589979 >> 172.21.15.49:0/542589979 conn(0x7f013c073480 msgr2=0x7f013c1602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.151+0000 7f0140a00640 1 -- 172.21.15.49:0/542589979 shutdown_connections 2024-04-15T15:18:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.151+0000 7f0140a00640 1 -- 172.21.15.49:0/542589979 wait complete. 2024-04-15T15:18:58.171 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:18:58.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.400+0000 7f7b46400640 1 Processor -- start 2024-04-15T15:18:58.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.401+0000 7f7b46400640 1 -- start start 2024-04-15T15:18:58.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.402+0000 7f7b46400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b4006fc80 0x7f7b4015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.402+0000 7f7b46400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7b40072150 0x7f7b400725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.402+0000 7f7b46400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b4015d520 con 0x7f7b4015a440 2024-04-15T15:18:58.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.402+0000 7f7b46400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7b40072b00 con 0x7f7b40072150 2024-04-15T15:18:58.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.402+0000 7f7b46400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4006e280 con 0x7f7b4006fc80 2024-04-15T15:18:58.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.403+0000 7f7b3f400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7b4015a440 0x7f7b400cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42932/0 (socket says 172.21.15.49:42932) 2024-04-15T15:18:58.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.403+0000 7f7b3f400640 1 -- 172.21.15.49:0/909886373 learned_addr learned my addr 172.21.15.49:0/909886373 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:58.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.403+0000 7f7b3fe00640 1 --2- 172.21.15.49:0/909886373 >> v2:172.21.15.49:3300/0 conn(0x7f7b40072150 0x7f7b400725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.403+0000 7f7b3ea00640 1 --2- 172.21.15.49:0/909886373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b4006fc80 0x7f7b4015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.403+0000 7f7b3fe00640 1 -- 172.21.15.49:0/909886373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b4006fc80 msgr2=0x7f7b4015cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:58.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.403+0000 7f7b3fe00640 1 --2- 172.21.15.49:0/909886373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b4006fc80 0x7f7b4015cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.403+0000 7f7b3fe00640 1 -- 172.21.15.49:0/909886373 >> v1:172.21.15.49:6789/0 conn(0x7f7b4015a440 legacy=0x7f7b400cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.403+0000 7f7b3fe00640 1 -- 172.21.15.49:0/909886373 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4015d800 con 0x7f7b40072150 2024-04-15T15:18:58.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.403+0000 7f7b3e000640 1 -- 172.21.15.49:0/909886373 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531677553 0 0) 0x7f7b4015d520 con 0x7f7b4015a440 2024-04-15T15:18:58.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.403+0000 7f7b3ea00640 1 --2- 172.21.15.49:0/909886373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b4006fc80 0x7f7b4015cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:58.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.404+0000 7f7b3fe00640 1 --2- 172.21.15.49:0/909886373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b40072150 0x7f7b400725c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009d90 tx=0x7f7b3802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccfb4bcdd8acb17 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:58.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.405+0000 7f7b3e000640 1 -- 172.21.15.49:0/909886373 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38039070 con 0x7f7b40072150 2024-04-15T15:18:58.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.405+0000 7f7b3e000640 1 -- 172.21.15.49:0/909886373 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3802cc60 con 0x7f7b40072150 2024-04-15T15:18:58.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.405+0000 7f7b3e000640 1 -- 172.21.15.49:0/909886373 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38034660 con 0x7f7b40072150 2024-04-15T15:18:58.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.405+0000 7f7b46400640 1 -- 172.21.15.49:0/909886373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b40072150 msgr2=0x7f7b400725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.405+0000 7f7b46400640 1 --2- 172.21.15.49:0/909886373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b40072150 0x7f7b400725c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009d90 tx=0x7f7b3802beb0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.405+0000 7f7b46400640 1 -- 172.21.15.49:0/909886373 shutdown_connections 2024-04-15T15:18:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.405+0000 7f7b46400640 1 --2- 172.21.15.49:0/909886373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b40072150 0x7f7b400725c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.405+0000 7f7b46400640 1 --2- 172.21.15.49:0/909886373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b4006fc80 0x7f7b4015cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.405+0000 7f7b46400640 1 -- 172.21.15.49:0/909886373 >> 172.21.15.49:0/909886373 conn(0x7f7b400b2460 msgr2=0x7f7b400b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.405+0000 7f7b46400640 1 -- 172.21.15.49:0/909886373 shutdown_connections 2024-04-15T15:18:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.406+0000 7f7b46400640 1 -- 172.21.15.49:0/909886373 wait complete. 2024-04-15T15:18:58.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.408+0000 7f7b46400640 1 Processor -- start 2024-04-15T15:18:58.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.408+0000 7f7b46400640 1 -- start start 2024-04-15T15:18:58.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.409+0000 7f7b46400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4006fc80 0x7f7b401592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.409+0000 7f7b46400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b40072150 0x7f7b40142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.409+0000 7f7b46400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b4015a440 0x7f7b4015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.409+0000 7f7b46400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4015a820 con 0x7f7b4006fc80 2024-04-15T15:18:58.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.409+0000 7f7b46400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b400cd7f0 con 0x7f7b4015a440 2024-04-15T15:18:58.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.409+0000 7f7b46400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4015ddc0 con 0x7f7b40072150 2024-04-15T15:18:58.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.409+0000 7f7b3f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4006fc80 0x7f7b401592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.409+0000 7f7b3f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4006fc80 0x7f7b401592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39954/0 (socket says 172.21.15.49:39954) 2024-04-15T15:18:58.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.409+0000 7f7b3f400640 1 -- 172.21.15.49:0/1497773963 learned_addr learned my addr 172.21.15.49:0/1497773963 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b3ea00640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b40072150 0x7f7b40142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b3fe00640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b4015a440 0x7f7b4015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b3f400640 1 -- 172.21.15.49:0/1497773963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b40072150 msgr2=0x7f7b40142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b3f400640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b40072150 0x7f7b40142f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b3f400640 1 -- 172.21.15.49:0/1497773963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b4015a440 msgr2=0x7f7b4015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b3f400640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b4015a440 0x7f7b4015f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b3f400640 1 -- 172.21.15.49:0/1497773963 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b401437e0 con 0x7f7b4006fc80 2024-04-15T15:18:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b3f400640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4006fc80 0x7f7b401592d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000c400 tx=0x7f7b3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b37400640 1 -- 172.21.15.49:0/1497773963 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30017070 con 0x7f7b4006fc80 2024-04-15T15:18:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b37400640 1 -- 172.21.15.49:0/1497773963 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b30005ce0 con 0x7f7b4006fc80 2024-04-15T15:18:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b37400640 1 -- 172.21.15.49:0/1497773963 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30007310 con 0x7f7b4006fc80 2024-04-15T15:18:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.410+0000 7f7b46400640 1 -- 172.21.15.49:0/1497773963 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b400a9980 con 0x7f7b4006fc80 2024-04-15T15:18:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.411+0000 7f7b46400640 1 -- 172.21.15.49:0/1497773963 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b400a9f40 con 0x7f7b4006fc80 2024-04-15T15:18:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.412+0000 7f7b46400640 1 -- 172.21.15.49:0/1497773963 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b40068c80 con 0x7f7b4006fc80 2024-04-15T15:18:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.413+0000 7f7b37400640 1 -- 172.21.15.49:0/1497773963 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b3001a070 con 0x7f7b4006fc80 2024-04-15T15:18:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.414+0000 7f7b37400640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b2006c8f0 0x7f7b2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.414+0000 7f7b37400640 1 -- 172.21.15.49:0/1497773963 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7b3008af90 con 0x7f7b4006fc80 2024-04-15T15:18:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.414+0000 7f7b3ea00640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b2006c8f0 0x7f7b2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.415+0000 7f7b3ea00640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b2006c8f0 0x7f7b2006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b4015fcd0 tx=0x7f7b2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:58.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.419+0000 7f7b37400640 1 -- 172.21.15.49:0/1497773963 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b300593b0 con 0x7f7b4006fc80 2024-04-15T15:18:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.623+0000 7f7b46400640 1 -- 172.21.15.49:0/1497773963 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b400aa180 con 0x7f7b4006fc80 2024-04-15T15:18:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.624+0000 7f7b37400640 1 -- 172.21.15.49:0/1497773963 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f7b30058cb0 con 0x7f7b4006fc80 2024-04-15T15:18:58.626 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:18:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 -- 172.21.15.49:0/1497773963 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b2006c8f0 msgr2=0x7f7b2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b2006c8f0 0x7f7b2006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b4015fcd0 tx=0x7f7b2c008040 comp rx=0 tx=0).stop 2024-04-15T15:18:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 -- 172.21.15.49:0/1497773963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4006fc80 msgr2=0x7f7b401592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4006fc80 0x7f7b401592d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000c400 tx=0x7f7b3000c8c0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 -- 172.21.15.49:0/1497773963 shutdown_connections 2024-04-15T15:18:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b2006c8f0 0x7f7b2006edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b40072150 0x7f7b40142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b4015a440 0x7f7b4015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 --2- 172.21.15.49:0/1497773963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4006fc80 0x7f7b401592d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 -- 172.21.15.49:0/1497773963 >> 172.21.15.49:0/1497773963 conn(0x7f7b400b2460 msgr2=0x7f7b4015ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 -- 172.21.15.49:0/1497773963 shutdown_connections 2024-04-15T15:18:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.627+0000 7f7b46400640 1 -- 172.21.15.49:0/1497773963 wait complete. 2024-04-15T15:18:58.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:18:58.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.870+0000 7f6c2e400640 1 Processor -- start 2024-04-15T15:18:58.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.870+0000 7f6c2e400640 1 -- start start 2024-04-15T15:18:58.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.872+0000 7f6c2e400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6c280cebb0 0x7f6c28167590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.872+0000 7f6c2e400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c280cd2d0 con 0x7f6c280725f0 2024-04-15T15:18:58.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.872+0000 7f6c2e400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c280cd5b0 con 0x7f6c281461c0 2024-04-15T15:18:58.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.872+0000 7f6c2e400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f6c280cf1d0 con 0x7f6c280cebb0 2024-04-15T15:18:58.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.872+0000 7f6c27e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6c280725f0 0x7f6c280ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42940/0 (socket says 172.21.15.49:42940) 2024-04-15T15:18:58.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.872+0000 7f6c27e00640 1 -- 172.21.15.49:0/1592344528 learned_addr learned my addr 172.21.15.49:0/1592344528 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:18:58.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.873+0000 7f6c2da00640 1 --2- 172.21.15.49:0/1592344528 >> v2:172.21.15.107:3300/0 conn(0x7f6c280cebb0 0x7f6c28167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.873+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1902848713 0 0) 0x7f6c280cd2d0 con 0x7f6c280725f0 2024-04-15T15:18:58.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.873+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c10003520 con 0x7f6c280725f0 2024-04-15T15:18:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.873+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 175436311 0 0) 0x7f6c280cd5b0 con 0x7f6c281461c0 2024-04-15T15:18:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.873+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c280cd2d0 con 0x7f6c281461c0 2024-04-15T15:18:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.873+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1404983772 0 0) 0x7f6c10003520 con 0x7f6c280725f0 2024-04-15T15:18:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.873+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 >> v2:172.21.15.107:3300/0 conn(0x7f6c280cebb0 msgr2=0x7f6c28167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.873+0000 7f6c27400640 1 --2- 172.21.15.49:0/1592344528 >> v2:172.21.15.107:3300/0 conn(0x7f6c280cebb0 0x7f6c28167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.873+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 >> v1:172.21.15.107:6789/0 conn(0x7f6c281461c0 legacy=0x7f6c28071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.873+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c280cd890 con 0x7f6c280725f0 2024-04-15T15:18:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f6c14001670 con 0x7f6c280725f0 2024-04-15T15:18:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 >> v1:172.21.15.49:6789/0 conn(0x7f6c280725f0 legacy=0x7f6c280ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c27400640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c10003b90 0x7f6c10003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c27400640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c27400640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c100051c0 0x7f6c1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1000c090 con 0x7f6c10004800 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1000c210 con 0x7f6c10003b90 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1000c410 con 0x7f6c100051c0 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c27e00640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c2d000640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c10003b90 0x7f6c10003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.874+0000 7f6c2da00640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c100051c0 0x7f6c1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.875+0000 7f6c27e00640 1 -- 172.21.15.49:0/1592344528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c100051c0 msgr2=0x7f6c1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.875+0000 7f6c27e00640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c100051c0 0x7f6c1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.875+0000 7f6c27e00640 1 -- 172.21.15.49:0/1592344528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c10003b90 msgr2=0x7f6c10003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.875+0000 7f6c27e00640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c10003b90 0x7f6c10003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.875+0000 7f6c27e00640 1 -- 172.21.15.49:0/1592344528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c10003520 con 0x7f6c10004800 2024-04-15T15:18:58.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.875+0000 7f6c2d000640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c10003b90 0x7f6c10003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:58.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.875+0000 7f6c2da00640 1 -- 172.21.15.49:0/1592344528 reap_dead start 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.876+0000 7f6c27e00640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c10004c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c0e0 tx=0x7f6c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64948e4917fbde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.876+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14007c40 con 0x7f6c10004800 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.876+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c14030070 con 0x7f6c10004800 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.876+0000 7f6c27400640 1 -- 172.21.15.49:0/1592344528 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14005d90 con 0x7f6c10004800 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.877+0000 7f6c2e400640 1 -- 172.21.15.49:0/1592344528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 msgr2=0x7f6c10004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.877+0000 7f6c2e400640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c10004c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c0e0 tx=0x7f6c10005830 comp rx=0 tx=0).stop 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.877+0000 7f6c2e400640 1 -- 172.21.15.49:0/1592344528 shutdown_connections 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.877+0000 7f6c2e400640 1 --2- 172.21.15.49:0/1592344528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c10004c80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.877+0000 7f6c2e400640 1 -- 172.21.15.49:0/1592344528 >> 172.21.15.49:0/1592344528 conn(0x7f6c2806b930 msgr2=0x7f6c2806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.877+0000 7f6c2e400640 1 -- 172.21.15.49:0/1592344528 shutdown_connections 2024-04-15T15:18:58.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.877+0000 7f6c2e400640 1 -- 172.21.15.49:0/1592344528 wait complete. 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.879+0000 7f6c2e400640 1 Processor -- start 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.879+0000 7f6c2e400640 1 -- start start 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.880+0000 7f6c2e400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c28155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.880+0000 7f6c2e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c281563b0 0x7f6c281f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.880+0000 7f6c2e400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c281f8330 0x7f6c281e9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.880+0000 7f6c2e400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c280ce4a0 con 0x7f6c10004800 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.880+0000 7f6c2e400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c280cd080 con 0x7f6c281563b0 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.880+0000 7f6c2e400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6c280ce620 con 0x7f6c281f8330 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.880+0000 7f6c2d000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c28155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.880+0000 7f6c2d000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c28155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39972/0 (socket says 172.21.15.49:39972) 2024-04-15T15:18:58.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.880+0000 7f6c2d000640 1 -- 172.21.15.49:0/744986770 learned_addr learned my addr 172.21.15.49:0/744986770 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:58.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c2da00640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c281f8330 0x7f6c281e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c2d000640 1 -- 172.21.15.49:0/744986770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c281f8330 msgr2=0x7f6c281e9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c2d000640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c281f8330 0x7f6c281e9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c2d000640 1 -- 172.21.15.49:0/744986770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c281563b0 msgr2=0x7f6c281f7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c2d000640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c281563b0 0x7f6c281f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c2d000640 1 -- 172.21.15.49:0/744986770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c28156830 con 0x7f6c10004800 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c2d000640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c28155e70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00e5a0 tx=0x7f6c1c00ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c24c00640 1 -- 172.21.15.49:0/744986770 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0052e0 con 0x7f6c10004800 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c24c00640 1 -- 172.21.15.49:0/744986770 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c007020 con 0x7f6c10004800 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c24c00640 1 -- 172.21.15.49:0/744986770 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0071d0 con 0x7f6c10004800 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.881+0000 7f6c2e400640 1 -- 172.21.15.49:0/744986770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c281411d0 con 0x7f6c10004800 2024-04-15T15:18:58.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.882+0000 7f6c2e400640 1 -- 172.21.15.49:0/744986770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c280b4db0 con 0x7f6c10004800 2024-04-15T15:18:58.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.883+0000 7f6c2e400640 1 -- 172.21.15.49:0/744986770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c280755c0 con 0x7f6c10004800 2024-04-15T15:18:58.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.884+0000 7f6c24c00640 1 -- 172.21.15.49:0/744986770 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c1c01a020 con 0x7f6c10004800 2024-04-15T15:18:58.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.885+0000 7f6c24c00640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c0806c8f0 0x7f6c0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:58.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.885+0000 7f6c24c00640 1 -- 172.21.15.49:0/744986770 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6c1c08b210 con 0x7f6c10004800 2024-04-15T15:18:58.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.885+0000 7f6c27e00640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c0806c8f0 0x7f6c0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:58.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.886+0000 7f6c27e00640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c0806c8f0 0x7f6c0806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c14034000 tx=0x7f6c14007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:58.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:58.889+0000 7f6c24c00640 1 -- 172.21.15.49:0/744986770 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c1c059640 con 0x7f6c10004800 2024-04-15T15:18:59.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.097+0000 7f6c2e400640 1 -- 172.21.15.49:0/744986770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6c28072e40 con 0x7f6c10004800 2024-04-15T15:18:59.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.098+0000 7f6c24c00640 1 -- 172.21.15.49:0/744986770 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f6c1c058f40 con 0x7f6c10004800 2024-04-15T15:18:59.100 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:18:59.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.100+0000 7f6c2e400640 1 -- 172.21.15.49:0/744986770 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c0806c8f0 msgr2=0x7f6c0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c0806c8f0 0x7f6c0806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c14034000 tx=0x7f6c14007000 comp rx=0 tx=0).stop 2024-04-15T15:18:59.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 -- 172.21.15.49:0/744986770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 msgr2=0x7f6c28155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c28155e70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00e5a0 tx=0x7f6c1c00ea60 comp rx=0 tx=0).stop 2024-04-15T15:18:59.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 -- 172.21.15.49:0/744986770 shutdown_connections 2024-04-15T15:18:59.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c0806c8f0 0x7f6c0806edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c281f8330 0x7f6c281e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c281563b0 0x7f6c281f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 --2- 172.21.15.49:0/744986770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c10004800 0x7f6c28155e70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 -- 172.21.15.49:0/744986770 >> 172.21.15.49:0/744986770 conn(0x7f6c2806b930 msgr2=0x7f6c2806f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:59.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 -- 172.21.15.49:0/744986770 shutdown_connections 2024-04-15T15:18:59.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.101+0000 7f6c2e400640 1 -- 172.21.15.49:0/744986770 wait complete. 2024-04-15T15:18:59.122 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:18:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.351+0000 7fef99a00640 1 Processor -- start 2024-04-15T15:18:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.351+0000 7fef99a00640 1 -- start start 2024-04-15T15:18:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.353+0000 7fef99a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fef94148130 0x7fef9415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.353+0000 7fef99a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 0x7fef94071f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.353+0000 7fef99a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fef94072490 0x7fef9406dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.353+0000 7fef99a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fef94148510 con 0x7fef94072490 2024-04-15T15:18:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.353+0000 7fef99a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fef9415fd70 con 0x7fef94148130 2024-04-15T15:18:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.353+0000 7fef99a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fef94162940 con 0x7fef94162380 2024-04-15T15:18:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.353+0000 7fef99000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fef94072490 0x7fef9406dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.354+0000 7fef99000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fef94072490 0x7fef9406dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39986/0 (socket says 172.21.15.49:39986) 2024-04-15T15:18:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.354+0000 7fef99000640 1 -- 172.21.15.49:0/3663697502 learned_addr learned my addr 172.21.15.49:0/3663697502 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.354+0000 7fef93e00640 1 --2- 172.21.15.49:0/3663697502 >> v2:172.21.15.107:3300/0 conn(0x7fef94148130 0x7fef9415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.354+0000 7fef93400640 1 --2- 172.21.15.49:0/3663697502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 0x7fef94071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.354+0000 7fef99000640 1 -- 172.21.15.49:0/3663697502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 msgr2=0x7fef94071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.354+0000 7fef99000640 1 --2- 172.21.15.49:0/3663697502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 0x7fef94071f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.354+0000 7fef99000640 1 -- 172.21.15.49:0/3663697502 >> v2:172.21.15.107:3300/0 conn(0x7fef94148130 msgr2=0x7fef9415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.354+0000 7fef99000640 1 --2- 172.21.15.49:0/3663697502 >> v2:172.21.15.107:3300/0 conn(0x7fef94148130 0x7fef9415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.354+0000 7fef99000640 1 -- 172.21.15.49:0/3663697502 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef94162c20 con 0x7fef94072490 2024-04-15T15:18:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.355+0000 7fef99000640 1 --2- 172.21.15.49:0/3663697502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94072490 0x7fef9406dcd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef94073010 tx=0x7fef8c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f840a3d9568b7a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.356+0000 7fef92a00640 1 -- 172.21.15.49:0/3663697502 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c02ca80 con 0x7fef94072490 2024-04-15T15:18:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.356+0000 7fef92a00640 1 -- 172.21.15.49:0/3663697502 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c02cc20 con 0x7fef94072490 2024-04-15T15:18:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.356+0000 7fef92a00640 1 -- 172.21.15.49:0/3663697502 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0343f0 con 0x7fef94072490 2024-04-15T15:18:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.356+0000 7fef99a00640 1 -- 172.21.15.49:0/3663697502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94072490 msgr2=0x7fef9406dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.356+0000 7fef99a00640 1 --2- 172.21.15.49:0/3663697502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94072490 0x7fef9406dcd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef94073010 tx=0x7fef8c02bee0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.357+0000 7fef99a00640 1 -- 172.21.15.49:0/3663697502 shutdown_connections 2024-04-15T15:18:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.357+0000 7fef99a00640 1 --2- 172.21.15.49:0/3663697502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94072490 0x7fef9406dcd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.357+0000 7fef99a00640 1 --2- 172.21.15.49:0/3663697502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 0x7fef94071f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.357+0000 7fef99a00640 1 --2- 172.21.15.49:0/3663697502 >> v2:172.21.15.107:3300/0 conn(0x7fef94148130 0x7fef9415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.357+0000 7fef99a00640 1 -- 172.21.15.49:0/3663697502 >> 172.21.15.49:0/3663697502 conn(0x7fef9406b930 msgr2=0x7fef9406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:59.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.357+0000 7fef99a00640 1 -- 172.21.15.49:0/3663697502 shutdown_connections 2024-04-15T15:18:59.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.357+0000 7fef99a00640 1 -- 172.21.15.49:0/3663697502 wait complete. 2024-04-15T15:18:59.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.359+0000 7fef99a00640 1 Processor -- start 2024-04-15T15:18:59.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.359+0000 7fef99a00640 1 -- start start 2024-04-15T15:18:59.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.360+0000 7fef99a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef94072490 0x7fef940cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.360+0000 7fef99a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94148130 0x7fef9412d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.360+0000 7fef99a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 0x7fef941f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.360+0000 7fef99a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef940aeb90 con 0x7fef94148130 2024-04-15T15:18:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.360+0000 7fef99a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fef941631e0 con 0x7fef94072490 2024-04-15T15:18:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.360+0000 7fef99a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fef940aed10 con 0x7fef94162380 2024-04-15T15:18:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef93400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94148130 0x7fef9412d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef93400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94148130 0x7fef9412d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39998/0 (socket says 172.21.15.49:39998) 2024-04-15T15:18:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef93400640 1 -- 172.21.15.49:0/3009541795 learned_addr learned my addr 172.21.15.49:0/3009541795 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef93e00640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef94072490 0x7fef940cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef99000640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 0x7fef941f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef93400640 1 -- 172.21.15.49:0/3009541795 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 msgr2=0x7fef941f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef93400640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 0x7fef941f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef93400640 1 -- 172.21.15.49:0/3009541795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef94072490 msgr2=0x7fef940cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef93400640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef94072490 0x7fef940cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef93400640 1 -- 172.21.15.49:0/3009541795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef940cdfd0 con 0x7fef94148130 2024-04-15T15:18:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.361+0000 7fef99000640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 0x7fef941f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:18:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.362+0000 7fef93400640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94148130 0x7fef9412d880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef8000d5f0 tx=0x7fef8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.362+0000 7fef8be00640 1 -- 172.21.15.49:0/3009541795 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800125f0 con 0x7fef94148130 2024-04-15T15:18:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.362+0000 7fef8be00640 1 -- 172.21.15.49:0/3009541795 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef80019070 con 0x7fef94148130 2024-04-15T15:18:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.362+0000 7fef8be00640 1 -- 172.21.15.49:0/3009541795 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800129d0 con 0x7fef94148130 2024-04-15T15:18:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.362+0000 7fef99a00640 1 -- 172.21.15.49:0/3009541795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef940ce560 con 0x7fef94148130 2024-04-15T15:18:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.362+0000 7fef99a00640 1 -- 172.21.15.49:0/3009541795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef940cfe70 con 0x7fef94148130 2024-04-15T15:18:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.363+0000 7fef99a00640 1 -- 172.21.15.49:0/3009541795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef94069320 con 0x7fef94148130 2024-04-15T15:18:59.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.365+0000 7fef8be00640 1 -- 172.21.15.49:0/3009541795 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef80012b70 con 0x7fef94148130 2024-04-15T15:18:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.365+0000 7fef8be00640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fef7406c8f0 0x7fef7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.366+0000 7fef8be00640 1 -- 172.21.15.49:0/3009541795 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fef8008bf50 con 0x7fef94148130 2024-04-15T15:18:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.366+0000 7fef93e00640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fef7406c8f0 0x7fef7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.366+0000 7fef93e00640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fef7406c8f0 0x7fef7406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef84006fd0 tx=0x7fef84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:59.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.370+0000 7fef8be00640 1 -- 172.21.15.49:0/3009541795 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef94069320 con 0x7fef94148130 2024-04-15T15:18:59.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.620+0000 7fef99a00640 1 -- 172.21.15.49:0/3009541795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef940755c0 con 0x7fef94148130 2024-04-15T15:18:59.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.621+0000 7fef8be00640 1 -- 172.21.15.49:0/3009541795 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fef8005a2e0 con 0x7fef94148130 2024-04-15T15:18:59.622 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:18:59.623 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/203558139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":203558139},{"type":"v1","addr":"172.21.15.107:6833","nonce":203558139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3198352923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3198352923},{"type":"v1","addr":"172.21.15.49:6836","nonce":3198352923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1425337425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1425337425},{"type":"v1","addr":"172.21.15.49:6837","nonce":1425337425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:18:45.325981+0000","modified":"2024-04-15T15:18:48.378286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2374398161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2374398161},{"type":"v1","addr":"172.21.15.107:6835","nonce":2374398161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-15T15:18:59.623 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-04-15T15:18:59.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.623+0000 7fef99a00640 1 -- 172.21.15.49:0/3009541795 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fef7406c8f0 msgr2=0x7fef7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.623+0000 7fef99a00640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fef7406c8f0 0x7fef7406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef84006fd0 tx=0x7fef84008040 comp rx=0 tx=0).stop 2024-04-15T15:18:59.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.624+0000 7fef99a00640 1 -- 172.21.15.49:0/3009541795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94148130 msgr2=0x7fef9412d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.624+0000 7fef99a00640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94148130 0x7fef9412d880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef8000d5f0 tx=0x7fef8000dab0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.624+0000 7fef99a00640 1 -- 172.21.15.49:0/3009541795 shutdown_connections 2024-04-15T15:18:59.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.624+0000 7fef99a00640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fef7406c8f0 0x7fef7406edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.624+0000 7fef99a00640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef94162380 0x7fef941f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.624+0000 7fef99a00640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef94148130 0x7fef9412d880 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.624+0000 7fef99a00640 1 --2- 172.21.15.49:0/3009541795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef94072490 0x7fef940cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.624+0000 7fef99a00640 1 -- 172.21.15.49:0/3009541795 >> 172.21.15.49:0/3009541795 conn(0x7fef9406b930 msgr2=0x7fef9406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:59.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.624+0000 7fef99a00640 1 -- 172.21.15.49:0/3009541795 shutdown_connections 2024-04-15T15:18:59.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.624+0000 7fef99a00640 1 -- 172.21.15.49:0/3009541795 wait complete. 2024-04-15T15:18:59.647 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:18:59.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.872+0000 7fb5f5800640 1 Processor -- start 2024-04-15T15:18:59.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.873+0000 7fb5f5800640 1 -- start start 2024-04-15T15:18:59.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.874+0000 7fb5f5800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb5f0162020 0x7fb5f0162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.874+0000 7fb5f5800640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5f00d0090 con 0x7fb5f00d1c90 2024-04-15T15:18:59.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.874+0000 7fb5f5800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5f00cfad0 con 0x7fb5f00cf110 2024-04-15T15:18:59.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.874+0000 7fb5f5800640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f0163440 con 0x7fb5f0162020 2024-04-15T15:18:59.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.874+0000 7fb5efe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb5f0162020 0x7fb5f0162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.875+0000 7fb5efe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb5f0162020 0x7fb5f0162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40000/0 (socket says 172.21.15.49:40000) 2024-04-15T15:18:59.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.875+0000 7fb5efe00640 1 -- 172.21.15.49:0/387548873 learned_addr learned my addr 172.21.15.49:0/387548873 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:59.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.875+0000 7fb5efe00640 1 -- 172.21.15.49:0/387548873 >> v1:172.21.15.107:6789/0 conn(0x7fb5f00cf110 legacy=0x7fb5f00d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:18:59.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.875+0000 7fb5efe00640 1 -- 172.21.15.49:0/387548873 >> v1:172.21.15.49:6789/0 conn(0x7fb5f00d1c90 legacy=0x7fb5f00d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.875+0000 7fb5efe00640 1 -- 172.21.15.49:0/387548873 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f00cfdb0 con 0x7fb5f0162020 2024-04-15T15:18:59.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.875+0000 7fb5eea00640 1 -- 172.21.15.49:0/387548873 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1099259828 0 0) 0x7fb5e8001490 con 0x7fb5f00d1c90 2024-04-15T15:18:59.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.876+0000 7fb5efe00640 1 --2- 172.21.15.49:0/387548873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f0162020 0x7fb5f0162f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009870 tx=0x7fb5e002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e982edfb7fa43edc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:59.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.877+0000 7fb5eea00640 1 -- 172.21.15.49:0/387548873 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0034070 con 0x7fb5f0162020 2024-04-15T15:18:59.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.877+0000 7fb5eea00640 1 -- 172.21.15.49:0/387548873 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e0030860 con 0x7fb5f0162020 2024-04-15T15:18:59.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.877+0000 7fb5eea00640 1 -- 172.21.15.49:0/387548873 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0030c30 con 0x7fb5f0162020 2024-04-15T15:18:59.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.877+0000 7fb5f5800640 1 -- 172.21.15.49:0/387548873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f0162020 msgr2=0x7fb5f0162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.877+0000 7fb5f5800640 1 --2- 172.21.15.49:0/387548873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f0162020 0x7fb5f0162f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009870 tx=0x7fb5e002b9a0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.877+0000 7fb5f5800640 1 -- 172.21.15.49:0/387548873 shutdown_connections 2024-04-15T15:18:59.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.877+0000 7fb5f5800640 1 --2- 172.21.15.49:0/387548873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f0162020 0x7fb5f0162f00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.877+0000 7fb5f5800640 1 -- 172.21.15.49:0/387548873 >> 172.21.15.49:0/387548873 conn(0x7fb5f006cf00 msgr2=0x7fb5f006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:18:59.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.877+0000 7fb5f5800640 1 -- 172.21.15.49:0/387548873 shutdown_connections 2024-04-15T15:18:59.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.877+0000 7fb5f5800640 1 -- 172.21.15.49:0/387548873 wait complete. 2024-04-15T15:18:59.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.879+0000 7fb5f5800640 1 Processor -- start 2024-04-15T15:18:59.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.879+0000 7fb5f5800640 1 -- start start 2024-04-15T15:18:59.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.880+0000 7fb5f5800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f00cf110 0x7fb5f0152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.880+0000 7fb5f5800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5f00d1c90 0x7fb5f01559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.880+0000 7fb5f5800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5f0162020 0x7fb5f00ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.880+0000 7fb5f5800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f00cf880 con 0x7fb5f00cf110 2024-04-15T15:18:59.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.880+0000 7fb5f5800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f00b7b30 con 0x7fb5f00d1c90 2024-04-15T15:18:59.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.880+0000 7fb5f5800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f00d2e70 con 0x7fb5f0162020 2024-04-15T15:18:59.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5efe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f00cf110 0x7fb5f0152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5efe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f00cf110 0x7fb5f0152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40008/0 (socket says 172.21.15.49:40008) 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5efe00640 1 -- 172.21.15.49:0/1155939702 learned_addr learned my addr 172.21.15.49:0/1155939702 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5ef400640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5f00d1c90 0x7fb5f01559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5f4e00640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5f0162020 0x7fb5f00ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5efe00640 1 -- 172.21.15.49:0/1155939702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5f0162020 msgr2=0x7fb5f00ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5efe00640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5f0162020 0x7fb5f00ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5efe00640 1 -- 172.21.15.49:0/1155939702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5f00d1c90 msgr2=0x7fb5f01559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5efe00640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5f00d1c90 0x7fb5f01559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5efe00640 1 -- 172.21.15.49:0/1155939702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f0153200 con 0x7fb5f00cf110 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5ef400640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5f00d1c90 0x7fb5f01559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.881+0000 7fb5f4e00640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5f0162020 0x7fb5f00ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:18:59.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.882+0000 7fb5efe00640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f00cf110 0x7fb5f0152c20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009870 tx=0x7fb5e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:59.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.882+0000 7fb5e7e00640 1 -- 172.21.15.49:0/1155939702 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0048070 con 0x7fb5f00cf110 2024-04-15T15:18:59.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.882+0000 7fb5e7e00640 1 -- 172.21.15.49:0/1155939702 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e0030660 con 0x7fb5f00cf110 2024-04-15T15:18:59.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.882+0000 7fb5e7e00640 1 -- 172.21.15.49:0/1155939702 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0034040 con 0x7fb5f00cf110 2024-04-15T15:18:59.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.882+0000 7fb5f5800640 1 -- 172.21.15.49:0/1155939702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f01e8500 con 0x7fb5f00cf110 2024-04-15T15:18:59.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.882+0000 7fb5f5800640 1 -- 172.21.15.49:0/1155939702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f01e8ac0 con 0x7fb5f00cf110 2024-04-15T15:18:59.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.883+0000 7fb5f5800640 1 -- 172.21.15.49:0/1155939702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f01e82f0 con 0x7fb5f00cf110 2024-04-15T15:18:59.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.885+0000 7fb5e7e00640 1 -- 172.21.15.49:0/1155939702 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5e0037070 con 0x7fb5f00cf110 2024-04-15T15:18:59.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.886+0000 7fb5e7e00640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb5d006c920 0x7fb5d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:18:59.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.886+0000 7fb5e7e00640 1 -- 172.21.15.49:0/1155939702 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb5e00adb10 con 0x7fb5f00cf110 2024-04-15T15:18:59.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.886+0000 7fb5ef400640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb5d006c920 0x7fb5d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:18:59.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.887+0000 7fb5ef400640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb5d006c920 0x7fb5d006ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb5f01e97f0 tx=0x7fb5dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:18:59.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:18:59.890+0000 7fb5e7e00640 1 -- 172.21.15.49:0/1155939702 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5e007c050 con 0x7fb5f00cf110 2024-04-15T15:19:00.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.102+0000 7fb5f5800640 1 -- 172.21.15.49:0/1155939702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5f00755c0 con 0x7fb5f00cf110 2024-04-15T15:19:00.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.103+0000 7fb5e7e00640 1 -- 172.21.15.49:0/1155939702 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7fb5e007b950 con 0x7fb5f00cf110 2024-04-15T15:19:00.105 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:00.106 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:00.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 -- 172.21.15.49:0/1155939702 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb5d006c920 msgr2=0x7fb5d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb5d006c920 0x7fb5d006ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb5f01e97f0 tx=0x7fb5dc008040 comp rx=0 tx=0).stop 2024-04-15T15:19:00.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 -- 172.21.15.49:0/1155939702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f00cf110 msgr2=0x7fb5f0152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f00cf110 0x7fb5f0152c20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009870 tx=0x7fb5e0004730 comp rx=0 tx=0).stop 2024-04-15T15:19:00.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 -- 172.21.15.49:0/1155939702 shutdown_connections 2024-04-15T15:19:00.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb5d006c920 0x7fb5d006ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5f0162020 0x7fb5f00ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5f00d1c90 0x7fb5f01559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 --2- 172.21.15.49:0/1155939702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f00cf110 0x7fb5f0152c20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 -- 172.21.15.49:0/1155939702 >> 172.21.15.49:0/1155939702 conn(0x7fb5f006cf00 msgr2=0x7fb5f0074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:00.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 -- 172.21.15.49:0/1155939702 shutdown_connections 2024-04-15T15:19:00.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.106+0000 7fb5f5800640 1 -- 172.21.15.49:0/1155939702 wait complete. 2024-04-15T15:19:00.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:00.130 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:00.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.368+0000 7f201b200640 1 Processor -- start 2024-04-15T15:19:00.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.368+0000 7f201b200640 1 -- start start 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.369+0000 7f201b200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f201415ecc0 0x7f20140d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.369+0000 7f201b200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f20140b3720 0x7f20140cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.369+0000 7f201b200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20140ba4d0 con 0x7f20140d3270 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.369+0000 7f201b200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f201415f0a0 con 0x7f20140b3720 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.369+0000 7f201b200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f20140ba280 con 0x7f201415ecc0 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.370+0000 7f2019200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f20140b3720 0x7f20140cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.370+0000 7f2019200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f20140b3720 0x7f20140cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40010/0 (socket says 172.21.15.49:40010) 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.370+0000 7f2019200640 1 -- 172.21.15.49:0/3403939485 learned_addr learned my addr 172.21.15.49:0/3403939485 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.370+0000 7f2013e00640 1 --2- 172.21.15.49:0/3403939485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f201415ecc0 0x7f20140d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.370+0000 7f2019200640 1 -- 172.21.15.49:0/3403939485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f201415ecc0 msgr2=0x7f20140d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.370+0000 7f2019200640 1 --2- 172.21.15.49:0/3403939485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f201415ecc0 0x7f20140d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.370+0000 7f2019200640 1 -- 172.21.15.49:0/3403939485 >> v1:172.21.15.107:6789/0 conn(0x7f20140d3270 legacy=0x7f20140b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.370+0000 7f2019200640 1 -- 172.21.15.49:0/3403939485 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20140ba7b0 con 0x7f20140b3720 2024-04-15T15:19:00.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.370+0000 7f2013e00640 1 --2- 172.21.15.49:0/3403939485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f201415ecc0 0x7f20140d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:00.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.371+0000 7f2019200640 1 --2- 172.21.15.49:0/3403939485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140b3720 0x7f20140cf6a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f200c009870 tx=0x7f200c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3142f0377e715fb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.372+0000 7f2012a00640 1 -- 172.21.15.49:0/3403939485 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c034070 con 0x7f20140b3720 2024-04-15T15:19:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.372+0000 7f2012a00640 1 -- 172.21.15.49:0/3403939485 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200c030840 con 0x7f20140b3720 2024-04-15T15:19:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.372+0000 7f2012a00640 1 -- 172.21.15.49:0/3403939485 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c030c10 con 0x7f20140b3720 2024-04-15T15:19:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.372+0000 7f201b200640 1 -- 172.21.15.49:0/3403939485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140b3720 msgr2=0x7f20140cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.372+0000 7f201b200640 1 --2- 172.21.15.49:0/3403939485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140b3720 0x7f20140cf6a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f200c009870 tx=0x7f200c02b980 comp rx=0 tx=0).stop 2024-04-15T15:19:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.372+0000 7f201b200640 1 -- 172.21.15.49:0/3403939485 shutdown_connections 2024-04-15T15:19:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.372+0000 7f201b200640 1 --2- 172.21.15.49:0/3403939485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140b3720 0x7f20140cf6a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.372+0000 7f201b200640 1 --2- 172.21.15.49:0/3403939485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f201415ecc0 0x7f20140d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.373+0000 7f201b200640 1 -- 172.21.15.49:0/3403939485 >> 172.21.15.49:0/3403939485 conn(0x7f2014073480 msgr2=0x7f2014073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.373+0000 7f201b200640 1 -- 172.21.15.49:0/3403939485 shutdown_connections 2024-04-15T15:19:00.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.373+0000 7f201b200640 1 -- 172.21.15.49:0/3403939485 wait complete. 2024-04-15T15:19:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.374+0000 7f201b200640 1 Processor -- start 2024-04-15T15:19:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.375+0000 7f201b200640 1 -- start start 2024-04-15T15:19:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.375+0000 7f201b200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20140b3720 0x7f20140cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.375+0000 7f201b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140d3270 0x7f201414cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f201b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201415ecc0 0x7f20140ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f201b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f20140b8aa0 con 0x7f20140d3270 2024-04-15T15:19:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f201b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20140b90d0 con 0x7f201415ecc0 2024-04-15T15:19:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f201b200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f20140b9d00 con 0x7f20140b3720 2024-04-15T15:19:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f2013400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140d3270 0x7f201414cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f2013400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140d3270 0x7f201414cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40012/0 (socket says 172.21.15.49:40012) 2024-04-15T15:19:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f2013400640 1 -- 172.21.15.49:0/587120287 learned_addr learned my addr 172.21.15.49:0/587120287 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f2019200640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201415ecc0 0x7f20140ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f2013e00640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20140b3720 0x7f20140cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f2013400640 1 -- 172.21.15.49:0/587120287 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20140b3720 msgr2=0x7f20140cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f2013400640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20140b3720 0x7f20140cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f2013400640 1 -- 172.21.15.49:0/587120287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201415ecc0 msgr2=0x7f20140ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f2013400640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201415ecc0 0x7f20140ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.376+0000 7f2013400640 1 -- 172.21.15.49:0/587120287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20140cc130 con 0x7f20140d3270 2024-04-15T15:19:00.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.377+0000 7f2013400640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140d3270 0x7f201414cfa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c450 tx=0x7f1ffc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:00.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.377+0000 7f200be00640 1 -- 172.21.15.49:0/587120287 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0052d0 con 0x7f20140d3270 2024-04-15T15:19:00.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.377+0000 7f200be00640 1 -- 172.21.15.49:0/587120287 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ffc005470 con 0x7f20140d3270 2024-04-15T15:19:00.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.377+0000 7f201b200640 1 -- 172.21.15.49:0/587120287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2014161010 con 0x7f20140d3270 2024-04-15T15:19:00.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.377+0000 7f200be00640 1 -- 172.21.15.49:0/587120287 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc012870 con 0x7f20140d3270 2024-04-15T15:19:00.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.377+0000 7f201b200640 1 -- 172.21.15.49:0/587120287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20141615d0 con 0x7f20140d3270 2024-04-15T15:19:00.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.378+0000 7f201b200640 1 -- 172.21.15.49:0/587120287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2014073be0 con 0x7f20140d3270 2024-04-15T15:19:00.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.379+0000 7f200be00640 1 -- 172.21.15.49:0/587120287 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ffc012a10 con 0x7f20140d3270 2024-04-15T15:19:00.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.380+0000 7f200be00640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff006c8f0 0x7f1ff006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.380+0000 7f200be00640 1 -- 172.21.15.49:0/587120287 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1ffc08b750 con 0x7f20140d3270 2024-04-15T15:19:00.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.380+0000 7f2013e00640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff006c8f0 0x7f1ff006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.381+0000 7f2013e00640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff006c8f0 0x7f1ff006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2004008ee0 tx=0x7f2004008430 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:00.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.385+0000 7f200be00640 1 -- 172.21.15.49:0/587120287 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2014073be0 con 0x7f20140d3270 2024-04-15T15:19:00.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.588+0000 7f201b200640 1 -- 172.21.15.49:0/587120287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f20140b6620 con 0x7f20140d3270 2024-04-15T15:19:00.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.589+0000 7f200be00640 1 -- 172.21.15.49:0/587120287 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f20140b6620 con 0x7f20140d3270 2024-04-15T15:19:00.591 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:00.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 -- 172.21.15.49:0/587120287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff006c8f0 msgr2=0x7f1ff006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff006c8f0 0x7f1ff006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2004008ee0 tx=0x7f2004008430 comp rx=0 tx=0).stop 2024-04-15T15:19:00.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 -- 172.21.15.49:0/587120287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140d3270 msgr2=0x7f201414cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140d3270 0x7f201414cfa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c450 tx=0x7f1ffc00c910 comp rx=0 tx=0).stop 2024-04-15T15:19:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 -- 172.21.15.49:0/587120287 shutdown_connections 2024-04-15T15:19:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff006c8f0 0x7f1ff006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201415ecc0 0x7f20140ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20140d3270 0x7f201414cfa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 --2- 172.21.15.49:0/587120287 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20140b3720 0x7f20140cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 -- 172.21.15.49:0/587120287 >> 172.21.15.49:0/587120287 conn(0x7f2014073480 msgr2=0x7f20140b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 -- 172.21.15.49:0/587120287 shutdown_connections 2024-04-15T15:19:00.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.592+0000 7f201b200640 1 -- 172.21.15.49:0/587120287 wait complete. 2024-04-15T15:19:00.613 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.841+0000 7fef1b000640 1 Processor -- start 2024-04-15T15:19:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.842+0000 7fef1b000640 1 -- start start 2024-04-15T15:19:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.843+0000 7fef1b000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 0x7fef1406f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.843+0000 7fef1b000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fef140b3dc0 0x7fef1406f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.843+0000 7fef1b000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fef140cd4e0 0x7fef140cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.843+0000 7fef1b000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fef1415f230 con 0x7fef140b3dc0 2024-04-15T15:19:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.843+0000 7fef1b000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fef1415f4d0 con 0x7fef140cd4e0 2024-04-15T15:19:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.843+0000 7fef1b000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fef1415f7b0 con 0x7fef14158f10 2024-04-15T15:19:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.843+0000 7fef19200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fef140b3dc0 0x7fef1406f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.843+0000 7fef19200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fef140b3dc0 0x7fef1406f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40014/0 (socket says 172.21.15.49:40014) 2024-04-15T15:19:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.843+0000 7fef19200640 1 -- 172.21.15.49:0/2970923729 learned_addr learned my addr 172.21.15.49:0/2970923729 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.844+0000 7fef1a600640 1 --2- 172.21.15.49:0/2970923729 >> v2:172.21.15.107:3300/0 conn(0x7fef140cd4e0 0x7fef140cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.844+0000 7fef19c00640 1 --2- 172.21.15.49:0/2970923729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 0x7fef1406f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.844+0000 7fef19200640 1 -- 172.21.15.49:0/2970923729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 msgr2=0x7fef1406f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.844+0000 7fef19200640 1 --2- 172.21.15.49:0/2970923729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 0x7fef1406f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.844+0000 7fef19200640 1 -- 172.21.15.49:0/2970923729 >> v2:172.21.15.107:3300/0 conn(0x7fef140cd4e0 msgr2=0x7fef140cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.844+0000 7fef19200640 1 --2- 172.21.15.49:0/2970923729 >> v2:172.21.15.107:3300/0 conn(0x7fef140cd4e0 0x7fef140cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.844+0000 7fef19200640 1 -- 172.21.15.49:0/2970923729 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1415fa90 con 0x7fef140b3dc0 2024-04-15T15:19:00.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.844+0000 7fef19c00640 1 --2- 172.21.15.49:0/2970923729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 0x7fef1406f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:00.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.845+0000 7fef19200640 1 --2- 172.21.15.49:0/2970923729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140b3dc0 0x7fef1406f570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef140702f0 tx=0x7fef0002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77b91514518c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:00.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.845+0000 7fef13e00640 1 -- 172.21.15.49:0/2970923729 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0002e070 con 0x7fef140b3dc0 2024-04-15T15:19:00.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.845+0000 7fef13e00640 1 -- 172.21.15.49:0/2970923729 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef00037070 con 0x7fef140b3dc0 2024-04-15T15:19:00.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.845+0000 7fef13e00640 1 -- 172.21.15.49:0/2970923729 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00031a00 con 0x7fef140b3dc0 2024-04-15T15:19:00.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.846+0000 7fef1b000640 1 -- 172.21.15.49:0/2970923729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140b3dc0 msgr2=0x7fef1406f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.846+0000 7fef1b000640 1 --2- 172.21.15.49:0/2970923729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140b3dc0 0x7fef1406f570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef140702f0 tx=0x7fef0002b880 comp rx=0 tx=0).stop 2024-04-15T15:19:00.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.846+0000 7fef1b000640 1 -- 172.21.15.49:0/2970923729 shutdown_connections 2024-04-15T15:19:00.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.846+0000 7fef1b000640 1 --2- 172.21.15.49:0/2970923729 >> v2:172.21.15.107:3300/0 conn(0x7fef140cd4e0 0x7fef140cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.846+0000 7fef1b000640 1 --2- 172.21.15.49:0/2970923729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140b3dc0 0x7fef1406f570 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.846+0000 7fef1b000640 1 --2- 172.21.15.49:0/2970923729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 0x7fef1406f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.846+0000 7fef1b000640 1 -- 172.21.15.49:0/2970923729 >> 172.21.15.49:0/2970923729 conn(0x7fef1406b930 msgr2=0x7fef1406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:00.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.846+0000 7fef1b000640 1 -- 172.21.15.49:0/2970923729 shutdown_connections 2024-04-15T15:19:00.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.846+0000 7fef1b000640 1 -- 172.21.15.49:0/2970923729 wait complete. 2024-04-15T15:19:00.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.848+0000 7fef1b000640 1 Processor -- start 2024-04-15T15:19:00.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.848+0000 7fef1b000640 1 -- start start 2024-04-15T15:19:00.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.849+0000 7fef1b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef140b3dc0 0x7fef14138570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.849+0000 7fef1b000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140cd4e0 0x7fef1414ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.849+0000 7fef1b000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 0x7fef1414d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.849+0000 7fef1b000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef14159470 con 0x7fef140cd4e0 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.849+0000 7fef1b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fef141592f0 con 0x7fef140b3dc0 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.849+0000 7fef1b000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fef140cde70 con 0x7fef14158f10 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.849+0000 7fef19200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140cd4e0 0x7fef1414ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.849+0000 7fef19200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140cd4e0 0x7fef1414ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40024/0 (socket says 172.21.15.49:40024) 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.849+0000 7fef19200640 1 -- 172.21.15.49:0/2579339638 learned_addr learned my addr 172.21.15.49:0/2579339638 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.849+0000 7fef1a600640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 0x7fef1414d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.850+0000 7fef19c00640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef140b3dc0 0x7fef14138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.850+0000 7fef19200640 1 -- 172.21.15.49:0/2579339638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 msgr2=0x7fef1414d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.850+0000 7fef19200640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 0x7fef1414d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.850+0000 7fef19200640 1 -- 172.21.15.49:0/2579339638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef140b3dc0 msgr2=0x7fef14138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:00.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.850+0000 7fef19200640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef140b3dc0 0x7fef14138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:00.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.850+0000 7fef19200640 1 -- 172.21.15.49:0/2579339638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1413ad60 con 0x7fef140cd4e0 2024-04-15T15:19:00.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.850+0000 7fef19c00640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef140b3dc0 0x7fef14138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:00.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.850+0000 7fef19200640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140cd4e0 0x7fef1414ced0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fef0002b8c0 tx=0x7fef000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:00.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.850+0000 7fef11600640 1 -- 172.21.15.49:0/2579339638 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00032880 con 0x7fef140cd4e0 2024-04-15T15:19:00.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.851+0000 7fef11600640 1 -- 172.21.15.49:0/2579339638 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef00032a40 con 0x7fef140cd4e0 2024-04-15T15:19:00.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.851+0000 7fef11600640 1 -- 172.21.15.49:0/2579339638 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00032e30 con 0x7fef140cd4e0 2024-04-15T15:19:00.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.851+0000 7fef1b000640 1 -- 172.21.15.49:0/2579339638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef14138ab0 con 0x7fef140cd4e0 2024-04-15T15:19:00.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.851+0000 7fef1b000640 1 -- 172.21.15.49:0/2579339638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1413b2b0 con 0x7fef140cd4e0 2024-04-15T15:19:00.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.852+0000 7fef1b000640 1 -- 172.21.15.49:0/2579339638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef140cca30 con 0x7fef140cd4e0 2024-04-15T15:19:00.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.853+0000 7fef11600640 1 -- 172.21.15.49:0/2579339638 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef000453f0 con 0x7fef140cd4e0 2024-04-15T15:19:00.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.854+0000 7fef11600640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feef406c8f0 0x7feef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:00.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.854+0000 7fef11600640 1 -- 172.21.15.49:0/2579339638 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fef000ada20 con 0x7fef140cd4e0 2024-04-15T15:19:00.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.854+0000 7fef19c00640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feef406c8f0 0x7feef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:00.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.855+0000 7fef19c00640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feef406c8f0 0x7feef406edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef1414da90 tx=0x7fef04007430 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:00.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:00.859+0000 7fef11600640 1 -- 172.21.15.49:0/2579339638 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef00040070 con 0x7fef140cd4e0 2024-04-15T15:19:01.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.065+0000 7fef1b000640 1 -- 172.21.15.49:0/2579339638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fef1406c830 con 0x7fef140cd4e0 2024-04-15T15:19:01.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.066+0000 7fef11600640 1 -- 172.21.15.49:0/2579339638 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fef0007bdb0 con 0x7fef140cd4e0 2024-04-15T15:19:01.068 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 -- 172.21.15.49:0/2579339638 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feef406c8f0 msgr2=0x7feef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feef406c8f0 0x7feef406edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef1414da90 tx=0x7fef04007430 comp rx=0 tx=0).stop 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 -- 172.21.15.49:0/2579339638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140cd4e0 msgr2=0x7fef1414ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140cd4e0 0x7fef1414ced0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fef0002b8c0 tx=0x7fef000046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 -- 172.21.15.49:0/2579339638 shutdown_connections 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feef406c8f0 0x7feef406edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef14158f10 0x7fef1414d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef140cd4e0 0x7fef1414ced0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 --2- 172.21.15.49:0/2579339638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef140b3dc0 0x7fef14138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 -- 172.21.15.49:0/2579339638 >> 172.21.15.49:0/2579339638 conn(0x7fef1406b930 msgr2=0x7fef140cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 -- 172.21.15.49:0/2579339638 shutdown_connections 2024-04-15T15:19:01.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.069+0000 7fef1b000640 1 -- 172.21.15.49:0/2579339638 wait complete. 2024-04-15T15:19:01.092 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:01.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.311+0000 7fa5e7200640 1 Processor -- start 2024-04-15T15:19:01.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.312+0000 7fa5e7200640 1 -- start start 2024-04-15T15:19:01.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.313+0000 7fa5e7200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa5e0145070 0x7fa5e0072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.313+0000 7fa5e7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0072690 0x7fa5e015f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.313+0000 7fa5e7200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e015ffe0 con 0x7fa5e0163520 2024-04-15T15:19:01.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.313+0000 7fa5e7200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e00afde0 con 0x7fa5e0145070 2024-04-15T15:19:01.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.313+0000 7fa5e7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e01627f0 con 0x7fa5e0072690 2024-04-15T15:19:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.314+0000 7fa5e5200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0072690 0x7fa5e015f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.314+0000 7fa5e5200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0072690 0x7fa5e015f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:43400/0 (socket says 172.21.15.49:43400) 2024-04-15T15:19:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.314+0000 7fa5e5200640 1 -- 172.21.15.49:0/1657732964 learned_addr learned my addr 172.21.15.49:0/1657732964 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.314+0000 7fa5dfe00640 1 --2- 172.21.15.49:0/1657732964 >> v2:172.21.15.107:3300/0 conn(0x7fa5e0145070 0x7fa5e0072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.315+0000 7fa5e5200640 1 -- 172.21.15.49:0/1657732964 >> v2:172.21.15.107:3300/0 conn(0x7fa5e0145070 msgr2=0x7fa5e0072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.315+0000 7fa5e5200640 1 --2- 172.21.15.49:0/1657732964 >> v2:172.21.15.107:3300/0 conn(0x7fa5e0145070 0x7fa5e0072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.315+0000 7fa5e5200640 1 -- 172.21.15.49:0/1657732964 >> v1:172.21.15.107:6789/0 conn(0x7fa5e0163520 legacy=0x7fa5e015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.315+0000 7fa5e5200640 1 -- 172.21.15.49:0/1657732964 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e0162a40 con 0x7fa5e0072690 2024-04-15T15:19:01.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.316+0000 7fa5e5200640 1 --2- 172.21.15.49:0/1657732964 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0072690 0x7fa5e015f8b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa5d80098a0 tx=0x7fa5d802b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f078aae34277e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:01.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.317+0000 7fa5dea00640 1 -- 172.21.15.49:0/1657732964 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8035020 con 0x7fa5e0072690 2024-04-15T15:19:01.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.317+0000 7fa5dea00640 1 -- 172.21.15.49:0/1657732964 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d8031640 con 0x7fa5e0072690 2024-04-15T15:19:01.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.317+0000 7fa5dea00640 1 -- 172.21.15.49:0/1657732964 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8031a10 con 0x7fa5e0072690 2024-04-15T15:19:01.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.317+0000 7fa5e7200640 1 -- 172.21.15.49:0/1657732964 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0072690 msgr2=0x7fa5e015f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.317+0000 7fa5e7200640 1 --2- 172.21.15.49:0/1657732964 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0072690 0x7fa5e015f8b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa5d80098a0 tx=0x7fa5d802b8e0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.317+0000 7fa5e7200640 1 -- 172.21.15.49:0/1657732964 shutdown_connections 2024-04-15T15:19:01.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.317+0000 7fa5e7200640 1 --2- 172.21.15.49:0/1657732964 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0072690 0x7fa5e015f8b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.317+0000 7fa5e7200640 1 --2- 172.21.15.49:0/1657732964 >> v2:172.21.15.107:3300/0 conn(0x7fa5e0145070 0x7fa5e0072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.317+0000 7fa5e7200640 1 -- 172.21.15.49:0/1657732964 >> 172.21.15.49:0/1657732964 conn(0x7fa5e00b2460 msgr2=0x7fa5e00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.318+0000 7fa5e7200640 1 -- 172.21.15.49:0/1657732964 shutdown_connections 2024-04-15T15:19:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.318+0000 7fa5e7200640 1 -- 172.21.15.49:0/1657732964 wait complete. 2024-04-15T15:19:01.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.319+0000 7fa5e7200640 1 Processor -- start 2024-04-15T15:19:01.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.319+0000 7fa5e7200640 1 -- start start 2024-04-15T15:19:01.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.320+0000 7fa5e7200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5e0072690 0x7fa5e015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.320+0000 7fa5e7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e0145070 0x7fa5e013c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.320+0000 7fa5e7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0163520 0x7fa5e013c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.320+0000 7fa5e7200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0145450 con 0x7fa5e0072690 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.320+0000 7fa5e7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0163000 con 0x7fa5e0145070 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.320+0000 7fa5e7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e00aedd0 con 0x7fa5e0163520 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5dfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5e0072690 0x7fa5e015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5dfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5e0072690 0x7fa5e015ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40028/0 (socket says 172.21.15.49:40028) 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5dfe00640 1 -- 172.21.15.49:0/2445179974 learned_addr learned my addr 172.21.15.49:0/2445179974 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5e5200640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0163520 0x7fa5e013c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5df400640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e0145070 0x7fa5e013c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5dfe00640 1 -- 172.21.15.49:0/2445179974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0163520 msgr2=0x7fa5e013c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5dfe00640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0163520 0x7fa5e013c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5dfe00640 1 -- 172.21.15.49:0/2445179974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e0145070 msgr2=0x7fa5e013c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5dfe00640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e0145070 0x7fa5e013c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5dfe00640 1 -- 172.21.15.49:0/2445179974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e013adf0 con 0x7fa5e0072690 2024-04-15T15:19:01.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5df400640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e0145070 0x7fa5e013c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:01.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.321+0000 7fa5dfe00640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5e0072690 0x7fa5e015ac50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e9b0 tx=0x7fa5d000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:01.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.322+0000 7fa5d7e00640 1 -- 172.21.15.49:0/2445179974 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0017070 con 0x7fa5e0072690 2024-04-15T15:19:01.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.322+0000 7fa5d7e00640 1 -- 172.21.15.49:0/2445179974 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d000c9a0 con 0x7fa5e0072690 2024-04-15T15:19:01.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.322+0000 7fa5d7e00640 1 -- 172.21.15.49:0/2445179974 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000cd90 con 0x7fa5e0072690 2024-04-15T15:19:01.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.322+0000 7fa5e7200640 1 -- 172.21.15.49:0/2445179974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e0139d50 con 0x7fa5e0072690 2024-04-15T15:19:01.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.322+0000 7fa5e7200640 1 -- 172.21.15.49:0/2445179974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e013a310 con 0x7fa5e0072690 2024-04-15T15:19:01.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.323+0000 7fa5e7200640 1 -- 172.21.15.49:0/2445179974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e0068c80 con 0x7fa5e0072690 2024-04-15T15:19:01.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.324+0000 7fa5d7e00640 1 -- 172.21.15.49:0/2445179974 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5d001a070 con 0x7fa5e0072690 2024-04-15T15:19:01.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.325+0000 7fa5d7e00640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.325+0000 7fa5d7e00640 1 -- 172.21.15.49:0/2445179974 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa5d008bb50 con 0x7fa5e0072690 2024-04-15T15:19:01.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.325+0000 7fa5df400640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.326+0000 7fa5df400640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00cf920 tx=0x7fa5c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:01.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.329+0000 7fa5d7e00640 1 -- 172.21.15.49:0/2445179974 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5d0059f80 con 0x7fa5e0072690 2024-04-15T15:19:01.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.533+0000 7fa5e7200640 1 -- 172.21.15.49:0/2445179974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa5e006f8d0 con 0x7fa5e0072690 2024-04-15T15:19:01.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.534+0000 7fa5d7e00640 1 -- 172.21.15.49:0/2445179974 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fa5d0059880 con 0x7fa5e0072690 2024-04-15T15:19:01.535 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.536+0000 7fa5e7200640 1 -- 172.21.15.49:0/2445179974 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa5bc06c8f0 msgr2=0x7fa5bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.536+0000 7fa5e7200640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00cf920 tx=0x7fa5c8009040 comp rx=0 tx=0).stop 2024-04-15T15:19:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.536+0000 7fa5e7200640 1 -- 172.21.15.49:0/2445179974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5e0072690 msgr2=0x7fa5e015ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.536+0000 7fa5e7200640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5e0072690 0x7fa5e015ac50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e9b0 tx=0x7fa5d000ee70 comp rx=0 tx=0).stop 2024-04-15T15:19:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.536+0000 7fa5e7200640 1 -- 172.21.15.49:0/2445179974 shutdown_connections 2024-04-15T15:19:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.536+0000 7fa5e7200640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa5bc06c8f0 0x7fa5bc06edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.536+0000 7fa5e7200640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5e0163520 0x7fa5e013c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.536+0000 7fa5e7200640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e0145070 0x7fa5e013c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.536+0000 7fa5e7200640 1 --2- 172.21.15.49:0/2445179974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5e0072690 0x7fa5e015ac50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.536+0000 7fa5e7200640 1 -- 172.21.15.49:0/2445179974 >> 172.21.15.49:0/2445179974 conn(0x7fa5e00b2460 msgr2=0x7fa5e00af230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.537+0000 7fa5e7200640 1 -- 172.21.15.49:0/2445179974 shutdown_connections 2024-04-15T15:19:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.537+0000 7fa5e7200640 1 -- 172.21.15.49:0/2445179974 wait complete. 2024-04-15T15:19:01.558 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:01.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.784+0000 7f4de7200640 1 Processor -- start 2024-04-15T15:19:01.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.785+0000 7f4de7200640 1 -- start start 2024-04-15T15:19:01.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.786+0000 7f4de7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 0x7f4de8162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.786+0000 7f4de7200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4de80d0090 con 0x7f4de80d1c90 2024-04-15T15:19:01.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.786+0000 7f4de7200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4de80cfad0 con 0x7f4de80cf110 2024-04-15T15:19:01.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.786+0000 7f4de7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f4de8163440 con 0x7f4de8162020 2024-04-15T15:19:01.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.787+0000 7f4de6800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4de80d1c90 0x7f4de80d3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42952/0 (socket says 172.21.15.49:42952) 2024-04-15T15:19:01.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.787+0000 7f4de6800640 1 -- 172.21.15.49:0/3346202361 learned_addr learned my addr 172.21.15.49:0/3346202361 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:01.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.787+0000 7f4de5e00640 1 --2- 172.21.15.49:0/3346202361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 0x7f4de8162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.787+0000 7f4de4a00640 1 -- 172.21.15.49:0/3346202361 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645408040 0 0) 0x7f4de0001110 con 0x7f4de80d1c90 2024-04-15T15:19:01.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.787+0000 7f4de4a00640 1 -- 172.21.15.49:0/3346202361 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dcc003520 con 0x7f4de80d1c90 2024-04-15T15:19:01.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.787+0000 7f4de5e00640 1 -- 172.21.15.49:0/3346202361 >> v1:172.21.15.107:6789/0 conn(0x7f4de80cf110 legacy=0x7f4de80d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.787+0000 7f4de4a00640 1 -- 172.21.15.49:0/3346202361 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4234621574 0 0) 0x7f4dcc003520 con 0x7f4de80d1c90 2024-04-15T15:19:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.788+0000 7f4de5e00640 1 -- 172.21.15.49:0/3346202361 >> v1:172.21.15.49:6789/0 conn(0x7f4de80d1c90 legacy=0x7f4de80d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.788+0000 7f4de5e00640 1 -- 172.21.15.49:0/3346202361 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de80cfdb0 con 0x7f4de8162020 2024-04-15T15:19:01.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.789+0000 7f4de5e00640 1 --2- 172.21.15.49:0/3346202361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 0x7f4de8162f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009dc0 tx=0x7f4dd802bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de8fd5e4700b3655 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.789+0000 7f4de4a00640 1 -- 172.21.15.49:0/3346202361 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8039070 con 0x7f4de8162020 2024-04-15T15:19:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.789+0000 7f4de4a00640 1 -- 172.21.15.49:0/3346202361 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd802c910 con 0x7f4de8162020 2024-04-15T15:19:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.789+0000 7f4de4a00640 1 -- 172.21.15.49:0/3346202361 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0001840 con 0x7f4de8162020 2024-04-15T15:19:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.790+0000 7f4de7200640 1 -- 172.21.15.49:0/3346202361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 msgr2=0x7f4de8162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.790+0000 7f4de7200640 1 --2- 172.21.15.49:0/3346202361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 0x7f4de8162f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009dc0 tx=0x7f4dd802bde0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.790+0000 7f4de7200640 1 -- 172.21.15.49:0/3346202361 shutdown_connections 2024-04-15T15:19:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.790+0000 7f4de7200640 1 --2- 172.21.15.49:0/3346202361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 0x7f4de8162f00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.790+0000 7f4de7200640 1 -- 172.21.15.49:0/3346202361 >> 172.21.15.49:0/3346202361 conn(0x7f4de806cf00 msgr2=0x7f4de806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.790+0000 7f4de7200640 1 -- 172.21.15.49:0/3346202361 shutdown_connections 2024-04-15T15:19:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.790+0000 7f4de7200640 1 -- 172.21.15.49:0/3346202361 wait complete. 2024-04-15T15:19:01.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.792+0000 7f4de7200640 1 Processor -- start 2024-04-15T15:19:01.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.792+0000 7f4de7200640 1 -- start start 2024-04-15T15:19:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.793+0000 7f4de7200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4de80cf110 0x7f4de8147090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.793+0000 7f4de7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de80d1c90 0x7f4de8144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.793+0000 7f4de7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 0x7f4de81452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.793+0000 7f4de7200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4de80cf880 con 0x7f4de80cf110 2024-04-15T15:19:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.793+0000 7f4de7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4de80b7b30 con 0x7f4de80d1c90 2024-04-15T15:19:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.793+0000 7f4de7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f4de80d2e70 con 0x7f4de8162020 2024-04-15T15:19:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4de80cf110 0x7f4de8147090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4de80cf110 0x7f4de8147090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40034/0 (socket says 172.21.15.49:40034) 2024-04-15T15:19:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de5e00640 1 -- 172.21.15.49:0/1292731996 learned_addr learned my addr 172.21.15.49:0/1292731996 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:01.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de5400640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de80d1c90 0x7f4de8144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de6800640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 0x7f4de81452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de5e00640 1 -- 172.21.15.49:0/1292731996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 msgr2=0x7f4de81452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de5e00640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 0x7f4de81452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de5e00640 1 -- 172.21.15.49:0/1292731996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de80d1c90 msgr2=0x7f4de8144d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de5e00640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de80d1c90 0x7f4de8144d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de5e00640 1 -- 172.21.15.49:0/1292731996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de8145a30 con 0x7f4de80cf110 2024-04-15T15:19:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de6800640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 0x7f4de81452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:01.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.794+0000 7f4de5e00640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4de80cf110 0x7f4de8147090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8037040 tx=0x7f4dd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.795+0000 7f4dde000640 1 -- 172.21.15.49:0/1292731996 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8039050 con 0x7f4de80cf110 2024-04-15T15:19:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.795+0000 7f4dde000640 1 -- 172.21.15.49:0/1292731996 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd803a040 con 0x7f4de80cf110 2024-04-15T15:19:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.795+0000 7f4dde000640 1 -- 172.21.15.49:0/1292731996 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8009450 con 0x7f4de80cf110 2024-04-15T15:19:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.795+0000 7f4de7200640 1 -- 172.21.15.49:0/1292731996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de8145d10 con 0x7f4de80cf110 2024-04-15T15:19:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.795+0000 7f4de7200640 1 -- 172.21.15.49:0/1292731996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de81462d0 con 0x7f4de80cf110 2024-04-15T15:19:01.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.796+0000 7f4dd7400640 1 -- 172.21.15.49:0/1292731996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db8005290 con 0x7f4de80cf110 2024-04-15T15:19:01.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.797+0000 7f4dde000640 1 -- 172.21.15.49:0/1292731996 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4dd8003bd0 con 0x7f4de80cf110 2024-04-15T15:19:01.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.799+0000 7f4dde000640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4dc406c8f0 0x7f4dc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:01.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.799+0000 7f4dde000640 1 -- 172.21.15.49:0/1292731996 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4dd80ae240 con 0x7f4de80cf110 2024-04-15T15:19:01.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.799+0000 7f4de5400640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4dc406c8f0 0x7f4dc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:01.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.800+0000 7f4de5400640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4dc406c8f0 0x7f4dc406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4de80ca830 tx=0x7f4dd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:01.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:01.805+0000 7f4dde000640 1 -- 172.21.15.49:0/1292731996 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4dd803d360 con 0x7f4de80cf110 2024-04-15T15:19:02.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.056+0000 7f4dd7400640 1 -- 172.21.15.49:0/1292731996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4db8005580 con 0x7f4de80cf110 2024-04-15T15:19:02.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.058+0000 7f4dde000640 1 -- 172.21.15.49:0/1292731996 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f4dd807c690 con 0x7f4de80cf110 2024-04-15T15:19:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:02.060 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/203558139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":203558139},{"type":"v1","addr":"172.21.15.107:6833","nonce":203558139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3198352923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3198352923},{"type":"v1","addr":"172.21.15.49:6836","nonce":3198352923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1425337425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1425337425},{"type":"v1","addr":"172.21.15.49:6837","nonce":1425337425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:18:45.325981+0000","modified":"2024-04-15T15:18:48.378286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2374398161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2374398161},{"type":"v1","addr":"172.21.15.107:6835","nonce":2374398161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-15T15:19:02.060 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-04-15T15:19:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.060+0000 7f4dd7400640 1 -- 172.21.15.49:0/1292731996 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4dc406c8f0 msgr2=0x7f4dc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.060+0000 7f4dd7400640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4dc406c8f0 0x7f4dc406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4de80ca830 tx=0x7f4dd0009040 comp rx=0 tx=0).stop 2024-04-15T15:19:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.060+0000 7f4dd7400640 1 -- 172.21.15.49:0/1292731996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4de80cf110 msgr2=0x7f4de8147090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.060+0000 7f4dd7400640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4de80cf110 0x7f4de8147090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8037040 tx=0x7f4dd8004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.060+0000 7f4dd7400640 1 -- 172.21.15.49:0/1292731996 shutdown_connections 2024-04-15T15:19:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.060+0000 7f4dd7400640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4dc406c8f0 0x7f4dc406edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.060+0000 7f4dd7400640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4de8162020 0x7f4de81452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.060+0000 7f4dd7400640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de80d1c90 0x7f4de8144d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.060+0000 7f4dd7400640 1 --2- 172.21.15.49:0/1292731996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4de80cf110 0x7f4de8147090 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.060+0000 7f4dd7400640 1 -- 172.21.15.49:0/1292731996 >> 172.21.15.49:0/1292731996 conn(0x7f4de806cf00 msgr2=0x7f4de8160da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.061+0000 7f4dd7400640 1 -- 172.21.15.49:0/1292731996 shutdown_connections 2024-04-15T15:19:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.061+0000 7f4dd7400640 1 -- 172.21.15.49:0/1292731996 wait complete. 2024-04-15T15:19:02.081 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:02.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.309+0000 7fc352200640 1 Processor -- start 2024-04-15T15:19:02.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.309+0000 7fc352200640 1 -- start start 2024-04-15T15:19:02.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.311+0000 7fc352200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc34c0cee00 0x7fc34c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.311+0000 7fc352200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc34c071f50 0x7fc34c0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.311+0000 7fc352200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc34c15dbb0 con 0x7fc34c15cbe0 2024-04-15T15:19:02.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.311+0000 7fc352200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc34c06f4f0 con 0x7fc34c071f50 2024-04-15T15:19:02.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.311+0000 7fc352200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc34c0728f0 con 0x7fc34c0cee00 2024-04-15T15:19:02.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.311+0000 7fc343e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc34c071f50 0x7fc34c0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.311+0000 7fc351800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc34c15cbe0 0x7fc34c15d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42956/0 (socket says 172.21.15.49:42956) 2024-04-15T15:19:02.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.311+0000 7fc351800640 1 -- 172.21.15.49:0/335305368 learned_addr learned my addr 172.21.15.49:0/335305368 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:02.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.311+0000 7fc350e00640 1 --2- 172.21.15.49:0/335305368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc34c0cee00 0x7fc34c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.312+0000 7fc343e00640 1 -- 172.21.15.49:0/335305368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc34c0cee00 msgr2=0x7fc34c06e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:02.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.312+0000 7fc343e00640 1 --2- 172.21.15.49:0/335305368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc34c0cee00 0x7fc34c06e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.312+0000 7fc343e00640 1 -- 172.21.15.49:0/335305368 >> v1:172.21.15.49:6789/0 conn(0x7fc34c15cbe0 legacy=0x7fc34c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.312+0000 7fc343e00640 1 -- 172.21.15.49:0/335305368 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c15d8d0 con 0x7fc34c071f50 2024-04-15T15:19:02.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.313+0000 7fc343e00640 1 --2- 172.21.15.49:0/335305368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 0x7fc34c0723b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc3340097d0 tx=0x7fc33402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5988df78ec17781 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.313+0000 7fc34be00640 1 -- 172.21.15.49:0/335305368 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334035020 con 0x7fc34c071f50 2024-04-15T15:19:02.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.313+0000 7fc34be00640 1 -- 172.21.15.49:0/335305368 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc334031810 con 0x7fc34c071f50 2024-04-15T15:19:02.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.313+0000 7fc34be00640 1 -- 172.21.15.49:0/335305368 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334031be0 con 0x7fc34c071f50 2024-04-15T15:19:02.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.314+0000 7fc352200640 1 -- 172.21.15.49:0/335305368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 msgr2=0x7fc34c0723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.314+0000 7fc352200640 1 --2- 172.21.15.49:0/335305368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 0x7fc34c0723b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc3340097d0 tx=0x7fc33402b8b0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.314+0000 7fc352200640 1 -- 172.21.15.49:0/335305368 shutdown_connections 2024-04-15T15:19:02.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.314+0000 7fc352200640 1 --2- 172.21.15.49:0/335305368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 0x7fc34c0723b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.314+0000 7fc352200640 1 --2- 172.21.15.49:0/335305368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc34c0cee00 0x7fc34c06e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.314+0000 7fc352200640 1 -- 172.21.15.49:0/335305368 >> 172.21.15.49:0/335305368 conn(0x7fc34c06b930 msgr2=0x7fc34c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.314+0000 7fc352200640 1 -- 172.21.15.49:0/335305368 shutdown_connections 2024-04-15T15:19:02.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.314+0000 7fc352200640 1 -- 172.21.15.49:0/335305368 wait complete. 2024-04-15T15:19:02.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.316+0000 7fc352200640 1 Processor -- start 2024-04-15T15:19:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.316+0000 7fc352200640 1 -- start start 2024-04-15T15:19:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.317+0000 7fc352200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 0x7fc34c14a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.317+0000 7fc352200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc34c0cee00 0x7fc34c160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.317+0000 7fc352200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc34c15cbe0 0x7fc34c155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.317+0000 7fc352200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c06e3a0 con 0x7fc34c071f50 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.317+0000 7fc352200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c06e0c0 con 0x7fc34c15cbe0 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.317+0000 7fc352200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc34c15dda0 con 0x7fc34c0cee00 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.317+0000 7fc350e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 0x7fc34c14a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.317+0000 7fc350e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 0x7fc34c14a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40052/0 (socket says 172.21.15.49:40052) 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.317+0000 7fc350e00640 1 -- 172.21.15.49:0/2287737877 learned_addr learned my addr 172.21.15.49:0/2287737877 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.318+0000 7fc351800640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc34c15cbe0 0x7fc34c155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.318+0000 7fc343e00640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc34c0cee00 0x7fc34c160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.318+0000 7fc350e00640 1 -- 172.21.15.49:0/2287737877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc34c0cee00 msgr2=0x7fc34c160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.318+0000 7fc350e00640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc34c0cee00 0x7fc34c160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.318+0000 7fc350e00640 1 -- 172.21.15.49:0/2287737877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc34c15cbe0 msgr2=0x7fc34c155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.318+0000 7fc350e00640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc34c15cbe0 0x7fc34c155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.318+0000 7fc350e00640 1 -- 172.21.15.49:0/2287737877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c15f2e0 con 0x7fc34c071f50 2024-04-15T15:19:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.318+0000 7fc351800640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc34c15cbe0 0x7fc34c155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:02.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.318+0000 7fc350e00640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 0x7fc34c14a170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00d830 tx=0x7fc33c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.319+0000 7fc349600640 1 -- 172.21.15.49:0/2287737877 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00b6c0 con 0x7fc34c071f50 2024-04-15T15:19:02.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.319+0000 7fc349600640 1 -- 172.21.15.49:0/2287737877 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33c019070 con 0x7fc34c071f50 2024-04-15T15:19:02.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.319+0000 7fc349600640 1 -- 172.21.15.49:0/2287737877 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00ba80 con 0x7fc34c071f50 2024-04-15T15:19:02.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.319+0000 7fc352200640 1 -- 172.21.15.49:0/2287737877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34c15f5c0 con 0x7fc34c071f50 2024-04-15T15:19:02.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.319+0000 7fc352200640 1 -- 172.21.15.49:0/2287737877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34c15fb80 con 0x7fc34c071f50 2024-04-15T15:19:02.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.320+0000 7fc352200640 1 -- 172.21.15.49:0/2287737877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34c0755c0 con 0x7fc34c071f50 2024-04-15T15:19:02.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.321+0000 7fc349600640 1 -- 172.21.15.49:0/2287737877 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc33c01e460 con 0x7fc34c071f50 2024-04-15T15:19:02.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.322+0000 7fc349600640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc32806c920 0x7fc32806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.322+0000 7fc349600640 1 -- 172.21.15.49:0/2287737877 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc33c08c210 con 0x7fc34c071f50 2024-04-15T15:19:02.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.322+0000 7fc343e00640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc32806c920 0x7fc32806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.323+0000 7fc343e00640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc32806c920 0x7fc32806ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc334036000 tx=0x7fc334008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.326+0000 7fc349600640 1 -- 172.21.15.49:0/2287737877 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc33c05a630 con 0x7fc34c071f50 2024-04-15T15:19:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.531+0000 7fc352200640 1 -- 172.21.15.49:0/2287737877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc34c15c010 con 0x7fc34c071f50 2024-04-15T15:19:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.533+0000 7fc349600640 1 -- 172.21.15.49:0/2287737877 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7fc33c059f30 con 0x7fc34c071f50 2024-04-15T15:19:02.535 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:02.535 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:02.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.535+0000 7fc352200640 1 -- 172.21.15.49:0/2287737877 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc32806c920 msgr2=0x7fc32806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.535+0000 7fc352200640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc32806c920 0x7fc32806ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc334036000 tx=0x7fc334008030 comp rx=0 tx=0).stop 2024-04-15T15:19:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.535+0000 7fc352200640 1 -- 172.21.15.49:0/2287737877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 msgr2=0x7fc34c14a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.535+0000 7fc352200640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 0x7fc34c14a170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00d830 tx=0x7fc33c00dcf0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.536+0000 7fc352200640 1 -- 172.21.15.49:0/2287737877 shutdown_connections 2024-04-15T15:19:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.536+0000 7fc352200640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc32806c920 0x7fc32806ede0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.536+0000 7fc352200640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc34c0cee00 0x7fc34c160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.536+0000 7fc352200640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc34c15cbe0 0x7fc34c155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.536+0000 7fc352200640 1 --2- 172.21.15.49:0/2287737877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc34c071f50 0x7fc34c14a170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.536+0000 7fc352200640 1 -- 172.21.15.49:0/2287737877 >> 172.21.15.49:0/2287737877 conn(0x7fc34c06b930 msgr2=0x7fc34c0cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.536+0000 7fc352200640 1 -- 172.21.15.49:0/2287737877 shutdown_connections 2024-04-15T15:19:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.536+0000 7fc352200640 1 -- 172.21.15.49:0/2287737877 wait complete. 2024-04-15T15:19:02.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:02.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.786+0000 7f19e1400640 1 Processor -- start 2024-04-15T15:19:02.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.786+0000 7f19e1400640 1 -- start start 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.788+0000 7f19e1400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f19dc0cee00 0x7f19dc06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.788+0000 7f19e1400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19dc071f50 0x7f19dc0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.788+0000 7f19e1400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19dc15cbe0 0x7f19dc15d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.788+0000 7f19e1400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f19dc06f4f0 con 0x7f19dc071f50 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.788+0000 7f19e1400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f19dc0728f0 con 0x7f19dc0cee00 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.788+0000 7f19e1400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc15dbb0 con 0x7f19dc15cbe0 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.788+0000 7f19d3400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19dc071f50 0x7f19dc0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.789+0000 7f19d3400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19dc071f50 0x7f19dc0723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40060/0 (socket says 172.21.15.49:40060) 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.789+0000 7f19d3400640 1 -- 172.21.15.49:0/1171124345 learned_addr learned my addr 172.21.15.49:0/1171124345 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.789+0000 7f19dbe00640 1 --2- 172.21.15.49:0/1171124345 >> v2:172.21.15.107:3300/0 conn(0x7f19dc0cee00 0x7f19dc06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.789+0000 7f19e0a00640 1 --2- 172.21.15.49:0/1171124345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19dc15cbe0 0x7f19dc15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.789+0000 7f19d3400640 1 -- 172.21.15.49:0/1171124345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19dc15cbe0 msgr2=0x7f19dc15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:02.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.789+0000 7f19d3400640 1 --2- 172.21.15.49:0/1171124345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19dc15cbe0 0x7f19dc15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.789+0000 7f19d3400640 1 -- 172.21.15.49:0/1171124345 >> v2:172.21.15.107:3300/0 conn(0x7f19dc0cee00 msgr2=0x7f19dc06e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:02.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.789+0000 7f19d3400640 1 --2- 172.21.15.49:0/1171124345 >> v2:172.21.15.107:3300/0 conn(0x7f19dc0cee00 0x7f19dc06e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.789+0000 7f19d3400640 1 -- 172.21.15.49:0/1171124345 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc15d8d0 con 0x7f19dc071f50 2024-04-15T15:19:02.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.791+0000 7f19d3400640 1 --2- 172.21.15.49:0/1171124345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc071f50 0x7f19dc0723b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f19dc070360 tx=0x7f19c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa89d303d2d485a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.791+0000 7f19db400640 1 -- 172.21.15.49:0/1171124345 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4035020 con 0x7f19dc071f50 2024-04-15T15:19:02.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.791+0000 7f19db400640 1 -- 172.21.15.49:0/1171124345 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c4031640 con 0x7f19dc071f50 2024-04-15T15:19:02.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.791+0000 7f19db400640 1 -- 172.21.15.49:0/1171124345 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c402d040 con 0x7f19dc071f50 2024-04-15T15:19:02.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.792+0000 7f19e1400640 1 -- 172.21.15.49:0/1171124345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc071f50 msgr2=0x7f19dc0723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.792+0000 7f19e1400640 1 --2- 172.21.15.49:0/1171124345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc071f50 0x7f19dc0723b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f19dc070360 tx=0x7f19c402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.792+0000 7f19e1400640 1 -- 172.21.15.49:0/1171124345 shutdown_connections 2024-04-15T15:19:02.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.792+0000 7f19e1400640 1 --2- 172.21.15.49:0/1171124345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19dc15cbe0 0x7f19dc15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.792+0000 7f19e1400640 1 --2- 172.21.15.49:0/1171124345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc071f50 0x7f19dc0723b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.792+0000 7f19e1400640 1 --2- 172.21.15.49:0/1171124345 >> v2:172.21.15.107:3300/0 conn(0x7f19dc0cee00 0x7f19dc06e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.792+0000 7f19e1400640 1 -- 172.21.15.49:0/1171124345 >> 172.21.15.49:0/1171124345 conn(0x7f19dc06b930 msgr2=0x7f19dc06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:02.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.792+0000 7f19e1400640 1 -- 172.21.15.49:0/1171124345 shutdown_connections 2024-04-15T15:19:02.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.792+0000 7f19e1400640 1 -- 172.21.15.49:0/1171124345 wait complete. 2024-04-15T15:19:02.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.794+0000 7f19e1400640 1 Processor -- start 2024-04-15T15:19:02.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.794+0000 7f19e1400640 1 -- start start 2024-04-15T15:19:02.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.795+0000 7f19e1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19dc071f50 0x7f19dc157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.795+0000 7f19e1400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19dc0cee00 0x7f19dc159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.795+0000 7f19e1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc15cbe0 0x7f19dc150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.795+0000 7f19e1400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc06e3a0 con 0x7f19dc15cbe0 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.795+0000 7f19e1400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc06e0c0 con 0x7f19dc071f50 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.795+0000 7f19e1400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc15d5b0 con 0x7f19dc0cee00 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.795+0000 7f19e0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc15cbe0 0x7f19dc150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.795+0000 7f19e0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc15cbe0 0x7f19dc150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40076/0 (socket says 172.21.15.49:40076) 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.795+0000 7f19e0a00640 1 -- 172.21.15.49:0/1903211045 learned_addr learned my addr 172.21.15.49:0/1903211045 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19dbe00640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19dc071f50 0x7f19dc157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19e0a00640 1 -- 172.21.15.49:0/1903211045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19dc0cee00 msgr2=0x7f19dc159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19d3400640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19dc0cee00 0x7f19dc159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19e0a00640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19dc0cee00 0x7f19dc159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19e0a00640 1 -- 172.21.15.49:0/1903211045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19dc071f50 msgr2=0x7f19dc157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19e0a00640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19dc071f50 0x7f19dc157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19e0a00640 1 -- 172.21.15.49:0/1903211045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc1eff70 con 0x7f19dc15cbe0 2024-04-15T15:19:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19dbe00640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19dc071f50 0x7f19dc157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19e0a00640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc15cbe0 0x7f19dc150f70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f19d4007e40 tx=0x7f19d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19d8c00640 1 -- 172.21.15.49:0/1903211045 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4014620 con 0x7f19dc15cbe0 2024-04-15T15:19:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19d8c00640 1 -- 172.21.15.49:0/1903211045 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d4019070 con 0x7f19dc15cbe0 2024-04-15T15:19:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19e1400640 1 -- 172.21.15.49:0/1903211045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19dc1f04f0 con 0x7f19dc15cbe0 2024-04-15T15:19:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.796+0000 7f19d8c00640 1 -- 172.21.15.49:0/1903211045 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4014a20 con 0x7f19dc15cbe0 2024-04-15T15:19:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.797+0000 7f19e1400640 1 -- 172.21.15.49:0/1903211045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19dc1f0ab0 con 0x7f19dc15cbe0 2024-04-15T15:19:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.798+0000 7f19e1400640 1 -- 172.21.15.49:0/1903211045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19dc0755c0 con 0x7f19dc15cbe0 2024-04-15T15:19:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.799+0000 7f19d8c00640 1 -- 172.21.15.49:0/1903211045 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19d4014bc0 con 0x7f19dc15cbe0 2024-04-15T15:19:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.799+0000 7f19d8c00640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f19b806c8f0 0x7f19b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.800+0000 7f19d8c00640 1 -- 172.21.15.49:0/1903211045 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f19d408c490 con 0x7f19dc15cbe0 2024-04-15T15:19:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.800+0000 7f19dbe00640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f19b806c8f0 0x7f19b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.800+0000 7f19dbe00640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f19b806c8f0 0x7f19b806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009ac0 tx=0x7f19cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:02.804+0000 7f19d8c00640 1 -- 172.21.15.49:0/1903211045 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19d405a820 con 0x7f19dc15cbe0 2024-04-15T15:19:03.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.051+0000 7f19e1400640 1 -- 172.21.15.49:0/1903211045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19dc156610 con 0x7f19dc15cbe0 2024-04-15T15:19:03.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.053+0000 7f19d8c00640 1 -- 172.21.15.49:0/1903211045 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f19d405a120 con 0x7f19dc15cbe0 2024-04-15T15:19:03.055 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:03.055 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/203558139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":203558139},{"type":"v1","addr":"172.21.15.107:6833","nonce":203558139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3198352923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3198352923},{"type":"v1","addr":"172.21.15.49:6836","nonce":3198352923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1425337425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1425337425},{"type":"v1","addr":"172.21.15.49:6837","nonce":1425337425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:18:45.325981+0000","modified":"2024-04-15T15:18:48.378286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2374398161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2374398161},{"type":"v1","addr":"172.21.15.107:6835","nonce":2374398161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-15T15:19:03.055 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-04-15T15:19:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.055+0000 7f19e1400640 1 -- 172.21.15.49:0/1903211045 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f19b806c8f0 msgr2=0x7f19b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.055+0000 7f19e1400640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f19b806c8f0 0x7f19b806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009ac0 tx=0x7f19cc008040 comp rx=0 tx=0).stop 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.055+0000 7f19e1400640 1 -- 172.21.15.49:0/1903211045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc15cbe0 msgr2=0x7f19dc150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.055+0000 7f19e1400640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc15cbe0 0x7f19dc150f70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f19d4007e40 tx=0x7f19d400e4b0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.056+0000 7f19e1400640 1 -- 172.21.15.49:0/1903211045 shutdown_connections 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.056+0000 7f19e1400640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f19b806c8f0 0x7f19b806edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.056+0000 7f19e1400640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19dc0cee00 0x7f19dc159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.056+0000 7f19e1400640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19dc15cbe0 0x7f19dc150f70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.056+0000 7f19e1400640 1 --2- 172.21.15.49:0/1903211045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19dc071f50 0x7f19dc157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.056+0000 7f19e1400640 1 -- 172.21.15.49:0/1903211045 >> 172.21.15.49:0/1903211045 conn(0x7f19dc06b930 msgr2=0x7f19dc06e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.056+0000 7f19e1400640 1 -- 172.21.15.49:0/1903211045 shutdown_connections 2024-04-15T15:19:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.056+0000 7f19e1400640 1 -- 172.21.15.49:0/1903211045 wait complete. 2024-04-15T15:19:03.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:18:45.325981+0000', 'modified': '2024-04-15T15:18:48.378286+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6835/2374398161', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 2374398161}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 2374398161}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4270, 'qdb_cluster': [4270]} 2024-04-15T15:19:03.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-15T15:19:03.481 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-15T15:19:03.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.716+0000 7fd8c5200640 1 Processor -- start 2024-04-15T15:19:03.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.716+0000 7fd8c5200640 1 -- start start 2024-04-15T15:19:03.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.718+0000 7fd8c5200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c013f6a0 0x7fd8c00ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.718+0000 7fd8c5200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c006ef20 con 0x7fd8c0072580 2024-04-15T15:19:03.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.718+0000 7fd8c5200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c015d200 con 0x7fd8c0072100 2024-04-15T15:19:03.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.718+0000 7fd8c5200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c00a95b0 con 0x7fd8c013f6a0 2024-04-15T15:19:03.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.718+0000 7fd8be800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd8c0072580 0x7fd8c006e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42966/0 (socket says 172.21.15.49:42966) 2024-04-15T15:19:03.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.718+0000 7fd8be800640 1 -- 172.21.15.49:0/2807416388 learned_addr learned my addr 172.21.15.49:0/2807416388 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:03.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.719+0000 7fd8bde00640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c013f6a0 0x7fd8c00ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.719+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1395430853 0 0) 0x7fd8c006ef20 con 0x7fd8c0072580 2024-04-15T15:19:03.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.719+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8a4003520 con 0x7fd8c0072580 2024-04-15T15:19:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.719+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1457789357 0 0) 0x7fd8a4003520 con 0x7fd8c0072580 2024-04-15T15:19:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.719+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c013f6a0 msgr2=0x7fd8c00ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.719+0000 7fd8bd400640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c013f6a0 0x7fd8c00ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.719+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 >> v1:172.21.15.107:6789/0 conn(0x7fd8c0072100 legacy=0x7fd8c006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.719+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c015d4e0 con 0x7fd8c0072580 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.719+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fd8b80019f0 con 0x7fd8c0072580 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.719+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 >> v1:172.21.15.49:6789/0 conn(0x7fd8c0072580 legacy=0x7fd8c006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8bd400640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8a40039e0 0x7fd8a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8bd400640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c013f6a0 0x7fd8a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8bd400640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a4004d40 0x7fd8a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a400b9c0 con 0x7fd8a4004d40 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a400bb40 con 0x7fd8a40039e0 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a400bd40 con 0x7fd8c013f6a0 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8be800640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a4004d40 0x7fd8a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8bde00640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8a40039e0 0x7fd8a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8b7e00640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c013f6a0 0x7fd8a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8be800640 1 -- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c013f6a0 msgr2=0x7fd8a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8be800640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c013f6a0 0x7fd8a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8be800640 1 -- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8a40039e0 msgr2=0x7fd8a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8be800640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8a40039e0 0x7fd8a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.720+0000 7fd8be800640 1 -- 172.21.15.49:0/2807416388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a4003520 con 0x7fd8a4004d40 2024-04-15T15:19:03.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.722+0000 7fd8be800640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a4004d40 0x7fd8a40051c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800a800 tx=0x7fd8a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ea595a5f79ee87 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.722+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b800a450 con 0x7fd8a4004d40 2024-04-15T15:19:03.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.722+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b8034650 con 0x7fd8a4004d40 2024-04-15T15:19:03.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.722+0000 7fd8bd400640 1 -- 172.21.15.49:0/2807416388 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b802ce60 con 0x7fd8a4004d40 2024-04-15T15:19:03.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.722+0000 7fd8c5200640 1 -- 172.21.15.49:0/2807416388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a4004d40 msgr2=0x7fd8a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.722+0000 7fd8c5200640 1 --2- 172.21.15.49:0/2807416388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a4004d40 0x7fd8a40051c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800a800 tx=0x7fd8a4005830 comp rx=0 tx=0).stop 2024-04-15T15:19:03.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.722+0000 7fd8be800640 1 -- 172.21.15.49:0/2807416388 reap_dead start 2024-04-15T15:19:03.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.722+0000 7fd8c5200640 1 -- 172.21.15.49:0/2807416388 shutdown_connections 2024-04-15T15:19:03.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.722+0000 7fd8c5200640 1 -- 172.21.15.49:0/2807416388 >> 172.21.15.49:0/2807416388 conn(0x7fd8c00b2460 msgr2=0x7fd8c00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.723+0000 7fd8c5200640 1 -- 172.21.15.49:0/2807416388 shutdown_connections 2024-04-15T15:19:03.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.723+0000 7fd8c5200640 1 -- 172.21.15.49:0/2807416388 wait complete. 2024-04-15T15:19:03.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.724+0000 7fd8c5200640 1 Processor -- start 2024-04-15T15:19:03.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.725+0000 7fd8c5200640 1 -- start start 2024-04-15T15:19:03.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.725+0000 7fd8c5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8a4004d40 0x7fd8c0156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.725+0000 7fd8c5200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c01563a0 0x7fd8c013dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.725+0000 7fd8c5200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c013e0f0 0x7fd8c01576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.725+0000 7fd8c5200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c00729f0 con 0x7fd8c013e0f0 2024-04-15T15:19:03.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.725+0000 7fd8c5200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0070400 con 0x7fd8a4004d40 2024-04-15T15:19:03.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.725+0000 7fd8c5200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c013fef0 con 0x7fd8c01563a0 2024-04-15T15:19:03.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8be800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c013e0f0 0x7fd8c01576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8be800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c013e0f0 0x7fd8c01576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40090/0 (socket says 172.21.15.49:40090) 2024-04-15T15:19:03.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8be800640 1 -- 172.21.15.49:0/2364635389 learned_addr learned my addr 172.21.15.49:0/2364635389 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:03.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8bde00640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8a4004d40 0x7fd8c0156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8b7e00640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c01563a0 0x7fd8c013dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8be800640 1 -- 172.21.15.49:0/2364635389 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c01563a0 msgr2=0x7fd8c013dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8be800640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c01563a0 0x7fd8c013dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8be800640 1 -- 172.21.15.49:0/2364635389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8a4004d40 msgr2=0x7fd8c0156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8be800640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8a4004d40 0x7fd8c0156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8be800640 1 -- 172.21.15.49:0/2364635389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c01405e0 con 0x7fd8c013e0f0 2024-04-15T15:19:03.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.726+0000 7fd8b7e00640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c01563a0 0x7fd8c013dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:03.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.727+0000 7fd8be800640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c013e0f0 0x7fd8c01576d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8000f80 tx=0x7fd8c015d3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.727+0000 7fd8b6000640 1 -- 172.21.15.49:0/2364635389 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8045070 con 0x7fd8c013e0f0 2024-04-15T15:19:03.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.727+0000 7fd8b6000640 1 -- 172.21.15.49:0/2364635389 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b803a040 con 0x7fd8c013e0f0 2024-04-15T15:19:03.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.727+0000 7fd8b6000640 1 -- 172.21.15.49:0/2364635389 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b800a450 con 0x7fd8c013e0f0 2024-04-15T15:19:03.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.727+0000 7fd8c5200640 1 -- 172.21.15.49:0/2364635389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c01408c0 con 0x7fd8c013e0f0 2024-04-15T15:19:03.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.727+0000 7fd8c5200640 1 -- 172.21.15.49:0/2364635389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c0140e80 con 0x7fd8c013e0f0 2024-04-15T15:19:03.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.730+0000 7fd8b6000640 1 -- 172.21.15.49:0/2364635389 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8b8048070 con 0x7fd8c013e0f0 2024-04-15T15:19:03.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.730+0000 7fd8b6000640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd89c06c8f0 0x7fd89c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.731+0000 7fd8b6000640 1 -- 172.21.15.49:0/2364635389 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd8b80ad6d0 con 0x7fd8c013e0f0 2024-04-15T15:19:03.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.731+0000 7fd8bde00640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd89c06c8f0 0x7fd89c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.731+0000 7fd8c7df9740 1 Processor -- start 2024-04-15T15:19:03.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.731+0000 7fd8bde00640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd89c06c8f0 0x7fd89c06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0009bc0 tx=0x7fd8b0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.731+0000 7fd8c7df9740 1 -- start start 2024-04-15T15:19:03.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.732+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3922735a0 0x55a3922926e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.732+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a39224c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.732+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a39224c7c0 0x55a39224a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.732+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a3921b19c0 con 0x55a39224c7c0 2024-04-15T15:19:03.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.732+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a392158560 con 0x55a3922735a0 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.732+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x55a3921ad980 con 0x55a392273980 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8be800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a39224c7c0 0x55a39224a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8be800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a39224c7c0 0x55a39224a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40104/0 (socket says 172.21.15.49:40104) 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8be800640 1 -- 172.21.15.49:0/2987002780 learned_addr learned my addr 172.21.15.49:0/2987002780 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8bde00640 1 --2- 172.21.15.49:0/2987002780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3922735a0 0x55a3922926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8b7e00640 1 --2- 172.21.15.49:0/2987002780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a39224c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8be800640 1 -- 172.21.15.49:0/2987002780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 msgr2=0x55a39224c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8be800640 1 --2- 172.21.15.49:0/2987002780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a39224c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8be800640 1 -- 172.21.15.49:0/2987002780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3922735a0 msgr2=0x55a3922926e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8be800640 1 --2- 172.21.15.49:0/2987002780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3922735a0 0x55a3922926e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8be800640 1 -- 172.21.15.49:0/2987002780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a391ec42d0 con 0x55a39224c7c0 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.733+0000 7fd8be800640 1 --2- 172.21.15.49:0/2987002780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a39224c7c0 0x55a39224a150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8083aa0 tx=0x7fd8b80019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0e026ed14e5033 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8aea00640 1 -- 172.21.15.49:0/2987002780 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b807b3f0 con 0x55a39224c7c0 2024-04-15T15:19:03.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8aea00640 1 -- 172.21.15.49:0/2987002780 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b807f7a0 con 0x55a39224c7c0 2024-04-15T15:19:03.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8aea00640 1 -- 172.21.15.49:0/2987002780 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8083fd0 con 0x55a39224c7c0 2024-04-15T15:19:03.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8c7df9740 1 -- 172.21.15.49:0/2987002780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a39224c7c0 msgr2=0x55a39224a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/2987002780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a39224c7c0 0x55a39224a150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8083aa0 tx=0x7fd8b80019c0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8c7df9740 1 -- 172.21.15.49:0/2987002780 shutdown_connections 2024-04-15T15:19:03.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/2987002780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a39224c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/2987002780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a39224c7c0 0x55a39224a150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/2987002780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3922735a0 0x55a3922926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8c7df9740 1 -- 172.21.15.49:0/2987002780 >> 172.21.15.49:0/2987002780 conn(0x55a392248860 msgr2=0x55a3922913e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8c7df9740 1 -- 172.21.15.49:0/2987002780 shutdown_connections 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.734+0000 7fd8c7df9740 1 -- 172.21.15.49:0/2987002780 wait complete. 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.735+0000 7fd8c7df9740 1 -- start start 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.735+0000 7fd8c7df9740 1 Processor -- start 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.736+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a3922735a0 0x55a392294360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.736+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392273980 0x55a39228e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.736+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a392262a60 0x55a39228e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.736+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a392158560 con 0x55a392273980 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.736+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a39216b920 con 0x55a392262a60 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.736+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x55a3921b19c0 con 0x55a3922735a0 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8c7df9740 10 client.0 _ll_register_callbacks cb 0x55a39216b310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8b7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392273980 0x55a39228e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8b7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392273980 0x55a39228e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40118/0 (socket says 172.21.15.49:40118) 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8b7e00640 1 -- 172.21.15.49:0/1543597656 learned_addr learned my addr 172.21.15.49:0/1543597656 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:03.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8bde00640 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a3922735a0 0x55a392294360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8be800640 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a392262a60 0x55a39228e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8b7e00640 1 -- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a3922735a0 msgr2=0x55a392294360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8b7e00640 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a3922735a0 0x55a392294360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8b7e00640 1 -- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a392262a60 msgr2=0x55a39228e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8b7e00640 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a392262a60 0x55a39228e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8b7e00640 1 -- 172.21.15.49:0/1543597656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a391f8e6b0 con 0x55a392273980 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8b7e00640 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392273980 0x55a39228e250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80102c0 tx=0x7fd8a8010780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a570ffa2a582a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.737+0000 7fd8a3400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8a3400640 1 -- 172.21.15.49:0/1543597656 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8005320 con 0x55a392273980 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8a3400640 1 -- 172.21.15.49:0/1543597656 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd8a8007050 con 0x55a392273980 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8c7df9740 1 -- 172.21.15.49:0/1543597656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b807f7a0 con 0x55a392273980 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8a3400640 1 -- 172.21.15.49:0/1543597656 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fd8a8007250 con 0x55a392273980 2024-04-15T15:19:03.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8c7df9740 10 client.4629 fetch_fsmap learned FSMap version 8 2024-04-15T15:19:03.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8c7df9740 1 -- 172.21.15.49:0/1543597656 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55a39228c970 con 0x55a392273980 2024-04-15T15:19:03.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8a3400640 1 -- 172.21.15.49:0/1543597656 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b807f7a0 con 0x55a392273980 2024-04-15T15:19:03.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8a3400640 1 -- 172.21.15.49:0/1543597656 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fd8a80073f0 con 0x55a392273980 2024-04-15T15:19:03.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8c7df9740 10 client.4629 fetch_fsmap finished waiting for FSMap version 8 2024-04-15T15:19:03.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8c7df9740 10 client.4629 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-15T15:19:03.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.738+0000 7fd8c7df9740 20 client.4629 populate_metadata read hostname 'smithi049' 2024-04-15T15:19:03.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.739+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:6834/2374398161,v1:172.21.15.107:6835/2374398161] conn(0x55a39228d4e0 0x55a392293b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.739+0000 7fd8c7df9740 4 client.4629 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:19:03.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.739+0000 7fd8c7df9740 1 -- 172.21.15.49:0/1543597656 --> [v2:172.21.15.107:6834/2374398161,v1:172.21.15.107:6835/2374398161] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a392157b70 con 0x55a39228d4e0 2024-04-15T15:19:03.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.739+0000 7fd8bde00640 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:6834/2374398161,v1:172.21.15.107:6835/2374398161] conn(0x55a39228d4e0 0x55a392293b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.740+0000 7fd8bde00640 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:6834/2374398161,v1:172.21.15.107:6835/2374398161] conn(0x55a39228d4e0 0x55a392293b60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89fbae8bad14b34 server_cookie=801c96572934491a in_seq=0 out_seq=0 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.740+0000 7fd8a3400640 10 client.4629 ms_handle_connect on v2:172.21.15.107:6834/2374398161 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8a3400640 1 -- 172.21.15.49:0/1543597656 <== mds.0 v2:172.21.15.107:6834/2374398161 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a392157b70 con 0x55a39228d4e0 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8a3400640 10 client.4629 handle_command_reply: tid=0 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 client.4629 shutdown 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 -- 172.21.15.49:0/1543597656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392273980 msgr2=0x55a39228e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392273980 0x55a39228e250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80102c0 tx=0x7fd8a8010780 comp rx=0 tx=0).stop 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 -- 172.21.15.49:0/1543597656 shutdown_connections 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:6834/2374398161,v1:172.21.15.107:6835/2374398161] conn(0x55a39228d4e0 0x55a392293b60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a392262a60 0x55a39228e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392273980 0x55a39228e250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8a3400640 0 client.4629 ms_handle_reset on v2:172.21.15.107:6834/2374398161 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/1543597656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a3922735a0 0x55a392294360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 -- 172.21.15.49:0/1543597656 >> 172.21.15.49:0/1543597656 conn(0x55a39224c7c0 msgr2=0x55a392246720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 -- 172.21.15.49:0/1543597656 shutdown_connections 2024-04-15T15:19:03.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 1 -- 172.21.15.49:0/1543597656 wait complete. 2024-04-15T15:19:03.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.742+0000 7fd8c7df9740 20 client.4629 trim_cache size 0 max 16384 2024-04-15T15:19:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.762+0000 7fd8c7df9740 1 Processor -- start 2024-04-15T15:19:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.762+0000 7fd8c7df9740 1 -- start start 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.763+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392262a60 0x55a39228b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.763+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3922735a0 0x55a39228b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.763+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a39228dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.763+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01fc0a0 con 0x55a392262a60 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.763+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01588c0 con 0x55a3922735a0 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.763+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c01fba80 con 0x55a392273980 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.763+0000 7fd8bde00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392262a60 0x55a39228b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.763+0000 7fd8bde00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392262a60 0x55a39228b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40120/0 (socket says 172.21.15.49:40120) 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.763+0000 7fd8bde00640 1 -- 172.21.15.49:0/3581457551 learned_addr learned my addr 172.21.15.49:0/3581457551 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.764+0000 7fd8bde00640 1 -- 172.21.15.49:0/3581457551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 msgr2=0x55a39228dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.764+0000 7fd8be800640 1 --2- 172.21.15.49:0/3581457551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a39228dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.764+0000 7fd8bde00640 1 --2- 172.21.15.49:0/3581457551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a39228dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.764+0000 7fd8bde00640 1 -- 172.21.15.49:0/3581457551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3922735a0 msgr2=0x55a39228b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.764+0000 7fd8bde00640 1 --2- 172.21.15.49:0/3581457551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3922735a0 0x55a39228b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.764+0000 7fd8bde00640 1 -- 172.21.15.49:0/3581457551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a392262240 con 0x55a392262a60 2024-04-15T15:19:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.764+0000 7fd8bde00640 1 --2- 172.21.15.49:0/3581457551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392262a60 0x55a39228b2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0004330 tx=0x7fd8b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4a68ca17225818 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.764+0000 7fd8ae000640 1 -- 172.21.15.49:0/3581457551 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0031070 con 0x55a392262a60 2024-04-15T15:19:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.764+0000 7fd8ae000640 1 -- 172.21.15.49:0/3581457551 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b0004420 con 0x55a392262a60 2024-04-15T15:19:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.764+0000 7fd8ae000640 1 -- 172.21.15.49:0/3581457551 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b001b070 con 0x55a392262a60 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.765+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3581457551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392262a60 msgr2=0x55a39228b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.765+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/3581457551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392262a60 0x55a39228b2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0004330 tx=0x7fd8b0017000 comp rx=0 tx=0).stop 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.765+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3581457551 shutdown_connections 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.765+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/3581457551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a39228dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.765+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/3581457551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3922735a0 0x55a39228b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.765+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/3581457551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a392262a60 0x55a39228b2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.765+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3581457551 >> 172.21.15.49:0/3581457551 conn(0x55a39224c7c0 msgr2=0x55a3922a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.765+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3581457551 shutdown_connections 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.765+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3581457551 wait complete. 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.766+0000 7fd8c7df9740 1 -- start start 2024-04-15T15:19:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.766+0000 7fd8c7df9740 1 Processor -- start 2024-04-15T15:19:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a392262a60 0x55a39228df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3922735a0 0x55a39230ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8c7df9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a3922b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a39228d360 con 0x55a3922735a0 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a3922a1490 con 0x55a392262a60 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8c7df9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x55a39228d8f0 con 0x55a392273980 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8c7df9740 10 client.0 _ll_register_callbacks cb 0x7fd8a8001760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8b7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3922735a0 0x55a39230ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8b7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3922735a0 0x55a39230ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40124/0 (socket says 172.21.15.49:40124) 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8b7e00640 1 -- 172.21.15.49:0/3491491650 learned_addr learned my addr 172.21.15.49:0/3491491650 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.767+0000 7fd8bde00640 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a392262a60 0x55a39228df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8be800640 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a3922b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8b7e00640 1 -- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 msgr2=0x55a3922b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8b7e00640 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a3922b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8b7e00640 1 -- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a392262a60 msgr2=0x55a39228df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8b7e00640 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a392262a60 0x55a39228df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8b7e00640 1 -- 172.21.15.49:0/3491491650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a39228eb30 con 0x55a3922735a0 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8b7e00640 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3922735a0 0x55a39230ae90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8011010 tx=0x7fd8a8001320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9146aeaca9d323ec server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8a3400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8a3400640 1 -- 172.21.15.49:0/3491491650 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8005320 con 0x55a3922735a0 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8a3400640 1 -- 172.21.15.49:0/3491491650 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd8a8007d70 con 0x55a3922735a0 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3491491650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b0004420 con 0x55a3922735a0 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.768+0000 7fd8a3400640 1 -- 172.21.15.49:0/3491491650 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fd8a8032050 con 0x55a3922735a0 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.769+0000 7fd8c7df9740 10 client.4635 fetch_fsmap learned FSMap version 8 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.769+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3491491650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55a392291270 con 0x55a3922735a0 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.769+0000 7fd8a3400640 1 -- 172.21.15.49:0/3491491650 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b0004420 con 0x55a3922735a0 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.769+0000 7fd8a3400640 1 -- 172.21.15.49:0/3491491650 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fd8a8035020 con 0x55a3922735a0 2024-04-15T15:19:03.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.769+0000 7fd8c7df9740 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2024-04-15T15:19:03.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.769+0000 7fd8c7df9740 10 client.4635 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-15T15:19:03.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.769+0000 7fd8c7df9740 20 client.4635 populate_metadata read hostname 'smithi049' 2024-04-15T15:19:03.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.769+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:6834/2374398161,v1:172.21.15.107:6835/2374398161] conn(0x55a39228ba30 0x55a3922aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:03.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.769+0000 7fd8c7df9740 4 client.4635 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-15T15:19:03.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.769+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3491491650 --> [v2:172.21.15.107:6834/2374398161,v1:172.21.15.107:6835/2374398161] -- command(tid 0: {"prefix": "status"}) -- 0x55a3921bf3e0 con 0x55a39228ba30 2024-04-15T15:19:03.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.770+0000 7fd8bde00640 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:6834/2374398161,v1:172.21.15.107:6835/2374398161] conn(0x55a39228ba30 0x55a3922aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:03.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.770+0000 7fd8bde00640 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:6834/2374398161,v1:172.21.15.107:6835/2374398161] conn(0x55a39228ba30 0x55a3922aa0a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da4e257b97c8ba43 server_cookie=93927ded9ecb5211 in_seq=0 out_seq=0 2024-04-15T15:19:03.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.770+0000 7fd8a3400640 10 client.4635 ms_handle_connect on v2:172.21.15.107:6834/2374398161 2024-04-15T15:19:03.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8a3400640 1 -- 172.21.15.49:0/3491491650 <== mds.0 v2:172.21.15.107:6834/2374398161 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55a3921bf3e0 con 0x55a39228ba30 2024-04-15T15:19:03.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8a3400640 10 client.4635 handle_command_reply: tid=0 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 client.4635 shutdown 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3491491650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3922735a0 msgr2=0x55a39230ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3922735a0 0x55a39230ae90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8011010 tx=0x7fd8a8001320 comp rx=0 tx=0).stop 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3491491650 shutdown_connections 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:6834/2374398161,v1:172.21.15.107:6835/2374398161] conn(0x55a39228ba30 0x55a3922aa0a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a392273980 0x55a3922b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3922735a0 0x55a39230ae90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 --2- 172.21.15.49:0/3491491650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a392262a60 0x55a39228df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3491491650 >> 172.21.15.49:0/3491491650 conn(0x55a39224c7c0 msgr2=0x55a392246720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8a3400640 0 client.4635 ms_handle_reset on v2:172.21.15.107:6834/2374398161 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3491491650 shutdown_connections 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.771+0000 7fd8c7df9740 1 -- 172.21.15.49:0/3491491650 wait complete. 2024-04-15T15:19:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c7df9740 20 client.4635 trim_cache size 0 max 16384 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c", 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4270, 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 17.91598707, 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 8, 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 22, 2024-04-15T15:19:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 22, 2024-04-15T15:19:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 25.812923145999999 2024-04-15T15:19:03.777 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:19:03.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c5200640 1 -- 172.21.15.49:0/2364635389 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd89c06c8f0 msgr2=0x7fd89c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c5200640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd89c06c8f0 0x7fd89c06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0009bc0 tx=0x7fd8b0009190 comp rx=0 tx=0).stop 2024-04-15T15:19:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c5200640 1 -- 172.21.15.49:0/2364635389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c013e0f0 msgr2=0x7fd8c01576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c5200640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c013e0f0 0x7fd8c01576d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8000f80 tx=0x7fd8c015d3d0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c5200640 1 -- 172.21.15.49:0/2364635389 shutdown_connections 2024-04-15T15:19:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c5200640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd89c06c8f0 0x7fd89c06edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c5200640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8c01563a0 0x7fd8c013dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c5200640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c013e0f0 0x7fd8c01576d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c5200640 1 --2- 172.21.15.49:0/2364635389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8a4004d40 0x7fd8c0156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.772+0000 7fd8c5200640 1 -- 172.21.15.49:0/2364635389 >> 172.21.15.49:0/2364635389 conn(0x7fd8c00b2460 msgr2=0x7fd8c015c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.773+0000 7fd8c5200640 1 -- 172.21.15.49:0/2364635389 shutdown_connections 2024-04-15T15:19:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:03.773+0000 7fd8c5200640 1 -- 172.21.15.49:0/2364635389 wait complete. 2024-04-15T15:19:03.793 INFO:teuthology.run_tasks:Running task sequential... 2024-04-15T15:19:03.802 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-15T15:19:03.802 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-15T15:19:03.803 INFO:teuthology.task.print:Enabling mgr modules 2024-04-15T15:19:03.803 INFO:teuthology.run_tasks:Running task kclient... 2024-04-15T15:19:03.826 INFO:tasks.kclient:Mounting kernel clients... 2024-04-15T15:19:03.827 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-15T15:19:03.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:03.827 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-04-15T15:19:03.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:03.887 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-04-15T15:19:04.000 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-04-15T15:19:04.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:19:04.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:04.006 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-04-15T15:19:04.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:04.064 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-04-15T15:19:04.174 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-04-15T15:19:04.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:19:04.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:04.178 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-15T15:19:04.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:04.237 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-04-15T15:19:04.336 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2024-04-15T15:19:04.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:19:04.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:04.340 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-15T15:19:04.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:04.393 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-04-15T15:19:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2024-04-15T15:19:04.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:19:04.492 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-15T15:19:04.492 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:19:04.492 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:04.493 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:19:04.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.721+0000 7ff260a00640 1 Processor -- start 2024-04-15T15:19:04.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.721+0000 7ff260a00640 1 -- start start 2024-04-15T15:19:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.722+0000 7ff260a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff25c0d1c90 0x7ff25c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.722+0000 7ff260a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 0x7ff25c0d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.722+0000 7ff260a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff25c0d0090 con 0x7ff25c162020 2024-04-15T15:19:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.722+0000 7ff260a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff25c163440 con 0x7ff25c0d1c90 2024-04-15T15:19:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.722+0000 7ff260a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c0b7b30 con 0x7ff25c0cf110 2024-04-15T15:19:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.723+0000 7ff25aa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff25c0d1c90 0x7ff25c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.723+0000 7ff25aa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff25c0d1c90 0x7ff25c162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50028/0 (socket says 172.21.15.49:50028) 2024-04-15T15:19:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.723+0000 7ff25aa00640 1 -- 172.21.15.49:0/3708289291 learned_addr learned my addr 172.21.15.49:0/3708289291 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.723+0000 7ff25b400640 1 --1- 172.21.15.49:0/3708289291 >> v1:172.21.15.49:6789/0 conn(0x7ff25c162020 0x7ff25c0d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48718/0 (socket says 172.21.15.49:48718) 2024-04-15T15:19:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.723+0000 7ff25be00640 1 --2- 172.21.15.49:0/3708289291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 0x7ff25c0d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.723+0000 7ff25aa00640 1 -- 172.21.15.49:0/3708289291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 msgr2=0x7ff25c0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:04.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.723+0000 7ff25aa00640 1 --2- 172.21.15.49:0/3708289291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 0x7ff25c0d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.723+0000 7ff25aa00640 1 -- 172.21.15.49:0/3708289291 >> v1:172.21.15.49:6789/0 conn(0x7ff25c162020 legacy=0x7ff25c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.723+0000 7ff25aa00640 1 -- 172.21.15.49:0/3708289291 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c0cfad0 con 0x7ff25c0d1c90 2024-04-15T15:19:04.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.723+0000 7ff25be00640 1 --2- 172.21.15.49:0/3708289291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 0x7ff25c0d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:04.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.724+0000 7ff25aa00640 1 --2- 172.21.15.49:0/3708289291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c0d1c90 0x7ff25c162f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff244009870 tx=0x7ff24402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfac7a914ac9499 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:04.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.725+0000 7ff25a000640 1 -- 172.21.15.49:0/3708289291 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244034070 con 0x7ff25c0d1c90 2024-04-15T15:19:04.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.725+0000 7ff25a000640 1 -- 172.21.15.49:0/3708289291 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff244030840 con 0x7ff25c0d1c90 2024-04-15T15:19:04.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.725+0000 7ff25a000640 1 -- 172.21.15.49:0/3708289291 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244030c10 con 0x7ff25c0d1c90 2024-04-15T15:19:04.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.725+0000 7ff260a00640 1 -- 172.21.15.49:0/3708289291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c0d1c90 msgr2=0x7ff25c162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.725+0000 7ff260a00640 1 --2- 172.21.15.49:0/3708289291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c0d1c90 0x7ff25c162f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff244009870 tx=0x7ff24402b980 comp rx=0 tx=0).stop 2024-04-15T15:19:04.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.725+0000 7ff260a00640 1 -- 172.21.15.49:0/3708289291 shutdown_connections 2024-04-15T15:19:04.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.725+0000 7ff260a00640 1 --2- 172.21.15.49:0/3708289291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 0x7ff25c0d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.725+0000 7ff260a00640 1 --2- 172.21.15.49:0/3708289291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c0d1c90 0x7ff25c162f00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.725+0000 7ff260a00640 1 -- 172.21.15.49:0/3708289291 >> 172.21.15.49:0/3708289291 conn(0x7ff25c06cf00 msgr2=0x7ff25c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:04.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.726+0000 7ff260a00640 1 -- 172.21.15.49:0/3708289291 shutdown_connections 2024-04-15T15:19:04.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.726+0000 7ff260a00640 1 -- 172.21.15.49:0/3708289291 wait complete. 2024-04-15T15:19:04.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.727+0000 7ff260a00640 1 Processor -- start 2024-04-15T15:19:04.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.727+0000 7ff260a00640 1 -- start start 2024-04-15T15:19:04.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.728+0000 7ff260a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 0x7ff25c1f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.728+0000 7ff260a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff25c0d1c90 0x7ff25c1f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.728+0000 7ff260a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c162020 0x7ff25c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.728+0000 7ff260a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0d2b90 con 0x7ff25c162020 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.728+0000 7ff260a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0cf880 con 0x7ff25c0d1c90 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.728+0000 7ff260a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c0d2e70 con 0x7ff25c0cf110 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.729+0000 7ff25be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c162020 0x7ff25c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.729+0000 7ff25be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c162020 0x7ff25c0cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50030/0 (socket says 172.21.15.49:50030) 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.729+0000 7ff25be00640 1 -- 172.21.15.49:0/968906977 learned_addr learned my addr 172.21.15.49:0/968906977 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.729+0000 7ff25b400640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 0x7ff25c1f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.729+0000 7ff25aa00640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff25c0d1c90 0x7ff25c1f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.729+0000 7ff25be00640 1 -- 172.21.15.49:0/968906977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 msgr2=0x7ff25c1f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.729+0000 7ff25be00640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 0x7ff25c1f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.729+0000 7ff25be00640 1 -- 172.21.15.49:0/968906977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff25c0d1c90 msgr2=0x7ff25c1f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.729+0000 7ff25be00640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff25c0d1c90 0x7ff25c1f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.729+0000 7ff25be00640 1 -- 172.21.15.49:0/968906977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c158b10 con 0x7ff25c162020 2024-04-15T15:19:04.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.730+0000 7ff25be00640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c162020 0x7ff25c0cd140 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c5f0 tx=0x7ff25400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:04.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.730+0000 7ff253400640 1 -- 172.21.15.49:0/968906977 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254015070 con 0x7ff25c162020 2024-04-15T15:19:04.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.730+0000 7ff253400640 1 -- 172.21.15.49:0/968906977 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff254011820 con 0x7ff25c162020 2024-04-15T15:19:04.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.730+0000 7ff253400640 1 -- 172.21.15.49:0/968906977 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254011c10 con 0x7ff25c162020 2024-04-15T15:19:04.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.730+0000 7ff260a00640 1 -- 172.21.15.49:0/968906977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c158df0 con 0x7ff25c162020 2024-04-15T15:19:04.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.730+0000 7ff260a00640 1 -- 172.21.15.49:0/968906977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c1593b0 con 0x7ff25c162020 2024-04-15T15:19:04.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.731+0000 7ff260a00640 1 -- 172.21.15.49:0/968906977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25c0755c0 con 0x7ff25c162020 2024-04-15T15:19:04.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.732+0000 7ff253400640 1 -- 172.21.15.49:0/968906977 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff254018070 con 0x7ff25c162020 2024-04-15T15:19:04.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.733+0000 7ff253400640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff23806c8f0 0x7ff23806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:04.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.733+0000 7ff253400640 1 -- 172.21.15.49:0/968906977 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff25408bab0 con 0x7ff25c162020 2024-04-15T15:19:04.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.733+0000 7ff25b400640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff23806c8f0 0x7ff23806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:04.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.734+0000 7ff25b400640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff23806c8f0 0x7ff23806edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0b8cf0 tx=0x7ff24c005b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:04.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.737+0000 7ff253400640 1 -- 172.21.15.49:0/968906977 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff254059e50 con 0x7ff25c162020 2024-04-15T15:19:04.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.990+0000 7ff260a00640 1 -- 172.21.15.49:0/968906977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff25c0b7650 con 0x7ff25c162020 2024-04-15T15:19:04.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.990+0000 7ff253400640 1 -- 172.21.15.49:0/968906977 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7ff254059750 con 0x7ff25c162020 2024-04-15T15:19:04.992 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:19:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 -- 172.21.15.49:0/968906977 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff23806c8f0 msgr2=0x7ff23806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff23806c8f0 0x7ff23806edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0b8cf0 tx=0x7ff24c005b70 comp rx=0 tx=0).stop 2024-04-15T15:19:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 -- 172.21.15.49:0/968906977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c162020 msgr2=0x7ff25c0cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c162020 0x7ff25c0cd140 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c5f0 tx=0x7ff25400cab0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 -- 172.21.15.49:0/968906977 shutdown_connections 2024-04-15T15:19:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff23806c8f0 0x7ff23806edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff25c162020 0x7ff25c0cd140 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff25c0d1c90 0x7ff25c1f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 --2- 172.21.15.49:0/968906977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25c0cf110 0x7ff25c1f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 -- 172.21.15.49:0/968906977 >> 172.21.15.49:0/968906977 conn(0x7ff25c06cf00 msgr2=0x7ff25c0cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.993+0000 7ff260a00640 1 -- 172.21.15.49:0/968906977 shutdown_connections 2024-04-15T15:19:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:04.994+0000 7ff260a00640 1 -- 172.21.15.49:0/968906977 wait complete. 2024-04-15T15:19:05.014 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:19:05.014 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:19:05.014 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-04-15T15:19:05.015 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-15T15:19:05.015 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:05.015 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:19:05.015 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:19:05.015 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:19:05.015 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:19:05.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:05.015 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-04-15T15:19:05.069 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:05.069 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:05.070 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-04-15T15:19:05.071 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-15T15:19:05.071 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42558sec preferred_lft 42558sec 2024-04-15T15:19:05.071 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-04-15T15:19:05.071 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:05.071 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:05.071 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:05.071 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-04-15T15:19:05.072 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-15T15:19:05.073 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:05.073 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:19:05.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2024-04-15T15:19:05.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2024-04-15T15:19:05.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2024-04-15T15:19:05.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-15T15:19:05.073 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:19:05.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:05.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:05.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:05.658 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:19:05.744 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-04-15T15:19:05.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:05.748 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-04-15T15:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-04-15T15:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-15T15:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-15T15:19:05.816 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:19:05.816 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:05.817 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:19:05.817 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-15T15:19:05.817 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-15T15:19:05.817 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-15T15:19:05.817 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:19:05.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:06.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:06.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:06.651 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-04-15T15:19:06.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:06.708 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-04-15T15:19:06.767 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:06.767 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:19:06.767 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:06.767 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-15T15:19:06.767 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:19:06.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:06.997 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-15T15:19:06.998 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:06.998 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:19:06.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-15T15:19:06.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-15T15:19:06.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-15T15:19:06.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-15T15:19:06.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-15T15:19:06.998 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:19:07.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:07.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:19:07.323 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:19:07.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:07.377 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:07.377 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:19:07.377 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-04-15T15:19:07.378 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2024-04-15T15:19:07.378 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:19:07.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:07.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:07.608 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:07.608 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:07.650 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:19:07.650 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:07.650 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:07.737 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:19:07.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:07.737 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:19:07.917 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:19:07.917 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:19:07.917 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:19:08.597 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:19:08.597 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:19:08.597 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:19:08.598 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:19:08.598 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:19:08.631 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:19:08.631 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:19:08.631 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:19:08.631 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:19:08.631 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:19:08.631 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790" 2024-04-15T15:19:08.631 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790:/ 2024-04-15T15:19:08.631 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c" will pass to kernel 2024-04-15T15:19:08.636 INFO:teuthology.orchestra.run.smithi049.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-15T15:19:08.636 INFO:teuthology.orchestra.run.smithi049.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:19:08.636 INFO:teuthology.orchestra.run.smithi049.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:19:08.636 INFO:teuthology.orchestra.run.smithi049.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:19:08.636 INFO:teuthology.orchestra.run.smithi049.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:19:08.643 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:19:08.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:08.643 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:19:08.728 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:08.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:08.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:08.872 INFO:teuthology.orchestra.run.smithi049.stdout:client4570 2024-04-15T15:19:08.872 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:19:08.872 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:08.920 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:09.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:09.103 INFO:teuthology.orchestra.run.smithi049.stdout:client4570 2024-04-15T15:19:09.103 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:09.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:09.253 INFO:teuthology.orchestra.run.smithi049.stdout:client4570 2024-04-15T15:19:09.253 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c.client4570/status') 2024-04-15T15:19:09.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:09.436 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.4570 (3)172.21.15.49:0/3276831583 2024-04-15T15:19:09.436 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2024-04-15T15:19:09.436 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2024-04-15T15:19:09.436 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2024-04-15T15:19:09.436 INFO:teuthology.orchestra.run.smithi049.stderr:70 bytes copied, 0.000353845 s, 198 kB/s 2024-04-15T15:19:09.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:09.451 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-15T15:19:09.451 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:19:09.451 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:09.451 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:19:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.678+0000 7f4bb0e00640 1 Processor -- start 2024-04-15T15:19:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.678+0000 7f4bb0e00640 1 -- start start 2024-04-15T15:19:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.680+0000 7f4bb0e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4bac072660 0x7f4bac06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.680+0000 7f4bb0e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bac15cbe0 0x7f4bac15d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.680+0000 7f4bb0e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bac15dbb0 con 0x7f4bac0cee00 2024-04-15T15:19:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.680+0000 7f4bb0e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f4bac06f4f0 con 0x7f4bac072660 2024-04-15T15:19:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.680+0000 7f4bb0e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac15d8d0 con 0x7f4bac15cbe0 2024-04-15T15:19:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.681+0000 7f4baaa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4bac072660 0x7f4bac06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.681+0000 7f4bab400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4bac0cee00 0x7f4bac071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48720/0 (socket says 172.21.15.49:48720) 2024-04-15T15:19:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.681+0000 7f4bab400640 1 -- 172.21.15.49:0/2893361990 learned_addr learned my addr 172.21.15.49:0/2893361990 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.681+0000 7f4babe00640 1 --2- 172.21.15.49:0/2893361990 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bac15cbe0 0x7f4bac15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.681+0000 7f4baaa00640 1 -- 172.21.15.49:0/2893361990 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bac15cbe0 msgr2=0x7f4bac15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.681+0000 7f4baaa00640 1 --2- 172.21.15.49:0/2893361990 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bac15cbe0 0x7f4bac15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.681+0000 7f4baaa00640 1 -- 172.21.15.49:0/2893361990 >> v1:172.21.15.49:6789/0 conn(0x7f4bac0cee00 legacy=0x7f4bac071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.681+0000 7f4baaa00640 1 -- 172.21.15.49:0/2893361990 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac06dde0 con 0x7f4bac072660 2024-04-15T15:19:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.681+0000 7f4baa000640 1 -- 172.21.15.49:0/2893361990 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1228283701 0 0) 0x7f4bac15dbb0 con 0x7f4bac0cee00 2024-04-15T15:19:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.681+0000 7f4babe00640 1 --2- 172.21.15.49:0/2893361990 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bac15cbe0 0x7f4bac15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.682+0000 7f4baaa00640 1 --2- 172.21.15.49:0/2893361990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac072660 0x7f4bac06e8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4b980097a0 tx=0x7f4b9802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3af68c0b35460d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:09.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.683+0000 7f4baa000640 1 -- 172.21.15.49:0/2893361990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98035020 con 0x7f4bac072660 2024-04-15T15:19:09.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.683+0000 7f4baa000640 1 -- 172.21.15.49:0/2893361990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b98031810 con 0x7f4bac072660 2024-04-15T15:19:09.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.683+0000 7f4baa000640 1 -- 172.21.15.49:0/2893361990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98031be0 con 0x7f4bac072660 2024-04-15T15:19:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.683+0000 7f4bb0e00640 1 -- 172.21.15.49:0/2893361990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac072660 msgr2=0x7f4bac06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.683+0000 7f4bb0e00640 1 --2- 172.21.15.49:0/2893361990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac072660 0x7f4bac06e8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4b980097a0 tx=0x7f4b9802b8b0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.683+0000 7f4bb0e00640 1 -- 172.21.15.49:0/2893361990 shutdown_connections 2024-04-15T15:19:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.683+0000 7f4bb0e00640 1 --2- 172.21.15.49:0/2893361990 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bac15cbe0 0x7f4bac15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.683+0000 7f4bb0e00640 1 --2- 172.21.15.49:0/2893361990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac072660 0x7f4bac06e8c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.683+0000 7f4bb0e00640 1 -- 172.21.15.49:0/2893361990 >> 172.21.15.49:0/2893361990 conn(0x7f4bac06b930 msgr2=0x7f4bac06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.684+0000 7f4bb0e00640 1 -- 172.21.15.49:0/2893361990 shutdown_connections 2024-04-15T15:19:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.684+0000 7f4bb0e00640 1 -- 172.21.15.49:0/2893361990 wait complete. 2024-04-15T15:19:09.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.685+0000 7f4bb0e00640 1 Processor -- start 2024-04-15T15:19:09.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.686+0000 7f4bb0e00640 1 -- start start 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.686+0000 7f4bb0e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bac072660 0x7f4bac146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.686+0000 7f4bb0e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac0cee00 0x7f4bac143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.686+0000 7f4bb0e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bac15cbe0 0x7f4bac155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4bb0e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac15d5b0 con 0x7f4bac0cee00 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4bb0e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac06e3a0 con 0x7f4bac15cbe0 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4bb0e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac15dda0 con 0x7f4bac072660 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4baaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac0cee00 0x7f4bac143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4baaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac0cee00 0x7f4bac143830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50040/0 (socket says 172.21.15.49:50040) 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4baaa00640 1 -- 172.21.15.49:0/3696679615 learned_addr learned my addr 172.21.15.49:0/3696679615 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4babe00640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bac15cbe0 0x7f4bac155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4baaa00640 1 -- 172.21.15.49:0/3696679615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bac072660 msgr2=0x7f4bac146af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4baaa00640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bac072660 0x7f4bac146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4baaa00640 1 -- 172.21.15.49:0/3696679615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bac15cbe0 msgr2=0x7f4bac155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4baaa00640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bac15cbe0 0x7f4bac155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.687+0000 7f4baaa00640 1 -- 172.21.15.49:0/3696679615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac147030 con 0x7f4bac0cee00 2024-04-15T15:19:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.688+0000 7f4baaa00640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac0cee00 0x7f4bac143830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4b980097a0 tx=0x7f4b980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.688+0000 7f4ba3400640 1 -- 172.21.15.49:0/3696679615 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98035050 con 0x7f4bac0cee00 2024-04-15T15:19:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.688+0000 7f4ba3400640 1 -- 172.21.15.49:0/3696679615 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9803bcf0 con 0x7f4bac0cee00 2024-04-15T15:19:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.688+0000 7f4ba3400640 1 -- 172.21.15.49:0/3696679615 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98004250 con 0x7f4bac0cee00 2024-04-15T15:19:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.688+0000 7f4bb0e00640 1 -- 172.21.15.49:0/3696679615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bac143d70 con 0x7f4bac0cee00 2024-04-15T15:19:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.688+0000 7f4bb0e00640 1 -- 172.21.15.49:0/3696679615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bac154840 con 0x7f4bac0cee00 2024-04-15T15:19:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.689+0000 7f4bb0e00640 1 -- 172.21.15.49:0/3696679615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bac0755c0 con 0x7f4bac0cee00 2024-04-15T15:19:09.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.690+0000 7f4ba3400640 1 -- 172.21.15.49:0/3696679615 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b98038020 con 0x7f4bac0cee00 2024-04-15T15:19:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.691+0000 7f4ba3400640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4b8c06c8f0 0x7f4b8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.691+0000 7f4ba3400640 1 -- 172.21.15.49:0/3696679615 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4b980adce0 con 0x7f4bac0cee00 2024-04-15T15:19:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.691+0000 7f4bab400640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4b8c06c8f0 0x7f4b8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.692+0000 7f4bab400640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4b8c06c8f0 0x7f4b8c06edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c007fd0 tx=0x7f4b9c007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.696+0000 7f4ba3400640 1 -- 172.21.15.49:0/3696679615 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bac143d70 con 0x7f4bac0cee00 2024-04-15T15:19:09.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.947+0000 7f4bb0e00640 1 -- 172.21.15.49:0/3696679615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4bac0b1d10 con 0x7f4bac0cee00 2024-04-15T15:19:09.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.948+0000 7f4ba3400640 1 -- 172.21.15.49:0/3696679615 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f4bac147030 con 0x7f4bac0cee00 2024-04-15T15:19:09.950 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:19:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.950+0000 7f4bb0e00640 1 -- 172.21.15.49:0/3696679615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4b8c06c8f0 msgr2=0x7f4b8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.950+0000 7f4bb0e00640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4b8c06c8f0 0x7f4b8c06edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c007fd0 tx=0x7f4b9c007920 comp rx=0 tx=0).stop 2024-04-15T15:19:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.950+0000 7f4bb0e00640 1 -- 172.21.15.49:0/3696679615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac0cee00 msgr2=0x7f4bac143830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.950+0000 7f4bb0e00640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac0cee00 0x7f4bac143830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4b980097a0 tx=0x7f4b980046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.951+0000 7f4bb0e00640 1 -- 172.21.15.49:0/3696679615 shutdown_connections 2024-04-15T15:19:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.951+0000 7f4bb0e00640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4b8c06c8f0 0x7f4b8c06edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.951+0000 7f4bb0e00640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bac15cbe0 0x7f4bac155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.951+0000 7f4bb0e00640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bac0cee00 0x7f4bac143830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.951+0000 7f4bb0e00640 1 --2- 172.21.15.49:0/3696679615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bac072660 0x7f4bac146af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:09.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.951+0000 7f4bb0e00640 1 -- 172.21.15.49:0/3696679615 >> 172.21.15.49:0/3696679615 conn(0x7f4bac06b930 msgr2=0x7f4bac156af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:09.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.951+0000 7f4bb0e00640 1 -- 172.21.15.49:0/3696679615 shutdown_connections 2024-04-15T15:19:09.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:09.951+0000 7f4bb0e00640 1 -- 172.21.15.49:0/3696679615 wait complete. 2024-04-15T15:19:09.971 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:19:09.971 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:19:09.971 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-04-15T15:19:09.971 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-15T15:19:09.972 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:09.972 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:19:09.972 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:19:09.972 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:19:09.972 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:19:09.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:09.972 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-04-15T15:19:10.027 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:10.027 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42553sec preferred_lft 42553sec 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:10.028 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 3a:d8:a3:bb:09:2d brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::6c47:44ff:feb2:eb9c/64 scope link 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 3a:d8:a3:bb:09:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::38d8:a3ff:febb:92d/64 scope link 2024-04-15T15:19:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:10.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:10.030 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:19:10.128 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-04-15T15:19:10.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:10.132 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-04-15T15:19:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-04-15T15:19:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:19:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-15T15:19:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-15T15:19:10.189 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:19:10.189 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:10.189 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:19:10.189 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-15T15:19:10.190 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-15T15:19:10.190 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-15T15:19:10.190 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:19:10.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:10.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:10.462 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-04-15T15:19:10.516 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:19:10.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:10.517 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-04-15T15:19:10.574 INFO:teuthology.orchestra.run.smithi049.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-15T15:19:10.575 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:10.575 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:19:10.575 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:19:10.575 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-15T15:19:10.576 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:19:10.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:10.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:10.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:10.798 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-15T15:19:10.901 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:10.901 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:10.901 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:10.901 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:10.901 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:10.902 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:10.902 INFO:teuthology.orchestra.run.smithi049.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:10.902 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether be:3f:5f:37:b7:40 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:19:10.902 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:19:10.902 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:10.902 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::bc3f:5fff:fe37:b740/64 scope link 2024-04-15T15:19:10.902 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:10.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:10.907 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-15T15:19:11.017 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:11.017 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:11.017 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:11.017 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:11.017 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:11.017 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:11.017 INFO:teuthology.orchestra.run.smithi049.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:11.017 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether be:3f:5f:37:b7:40 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:19:11.017 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:19:11.018 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:11.018 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::bc3f:5fff:fe37:b740/64 scope link 2024-04-15T15:19:11.018 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:11.023 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-15T15:19:11.023 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:11.023 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:19:11.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-15T15:19:11.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-15T15:19:11.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-15T15:19:11.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-15T15:19:11.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-15T15:19:11.023 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:19:11.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:11.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:11.405 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:11.405 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:19:11.405 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.1 up 2024-04-15T15:19:11.405 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.1 master ceph-brx 2024-04-15T15:19:11.406 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:19:11.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:11.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:11.656 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:11.656 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:11.712 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:19:11.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:11.713 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:11.769 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:19:11.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:11.769 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:19:11.927 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:19:11.927 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:19:11.927 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:19:11.950 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:19:11.950 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:19:11.950 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:19:11.950 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:19:11.950 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:19:11.981 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:19:11.982 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:19:11.982 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:19:11.982 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:19:11.982 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:19:11.982 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790" 2024-04-15T15:19:11.982 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790:/ 2024-04-15T15:19:11.982 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c" will pass to kernel 2024-04-15T15:19:11.984 INFO:teuthology.orchestra.run.smithi049.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-15T15:19:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:19:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:19:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:19:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:19:11.990 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:19:11.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:11.990 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:19:12.087 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:12.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:12.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:12.218 INFO:teuthology.orchestra.run.smithi049.stdout:client4439 2024-04-15T15:19:12.218 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:19:12.218 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:12.275 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:12.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:12.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:12.405 INFO:teuthology.orchestra.run.smithi049.stdout:client4439 2024-04-15T15:19:12.405 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:12.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:12.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:client4439 2024-04-15T15:19:12.525 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c.client4439/status') 2024-04-15T15:19:12.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:12.682 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.4439 (3)172.21.15.49:0/2327252612 2024-04-15T15:19:12.683 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2024-04-15T15:19:12.683 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2024-04-15T15:19:12.683 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2024-04-15T15:19:12.683 INFO:teuthology.orchestra.run.smithi049.stderr:70 bytes copied, 7.1401e-05 s, 980 kB/s 2024-04-15T15:19:12.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:12.697 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-15T15:19:12.697 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:19:12.697 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:12.697 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:19:12.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.919+0000 7f2470200640 1 Processor -- start 2024-04-15T15:19:12.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.920+0000 7f2470200640 1 -- start start 2024-04-15T15:19:12.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.921+0000 7f2470200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f246815b6d0 0x7f24680cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:12.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.921+0000 7f2470200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2468070210 0x7f24680b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:12.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.922+0000 7f2470200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f246806f940 con 0x7f24680ce250 2024-04-15T15:19:12.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.922+0000 7f2470200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f246806e8a0 con 0x7f246815b6d0 2024-04-15T15:19:12.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.922+0000 7f2470200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f246806eb80 con 0x7f2468070210 2024-04-15T15:19:12.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.922+0000 7f246d800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f246815b6d0 0x7f24680cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:12.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.922+0000 7f246e200640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f24680ce250 0x7f24680721e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48736/0 (socket says 172.21.15.49:48736) 2024-04-15T15:19:12.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.922+0000 7f246e200640 1 -- 172.21.15.49:0/3688425425 learned_addr learned my addr 172.21.15.49:0/3688425425 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:12.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.922+0000 7f246d800640 1 -- 172.21.15.49:0/3688425425 >> v2:172.21.15.107:3300/0 conn(0x7f2468070210 msgr2=0x7f24680b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:12.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.922+0000 7f246d800640 1 --2- 172.21.15.49:0/3688425425 >> v2:172.21.15.107:3300/0 conn(0x7f2468070210 0x7f24680b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:12.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.922+0000 7f246d800640 1 -- 172.21.15.49:0/3688425425 >> v1:172.21.15.49:6789/0 conn(0x7f24680ce250 legacy=0x7f24680721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:12.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.922+0000 7f246d800640 1 -- 172.21.15.49:0/3688425425 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246806ee60 con 0x7f246815b6d0 2024-04-15T15:19:12.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.924+0000 7f246d800640 1 --2- 172.21.15.49:0/3688425425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f246815b6d0 0x7f24680cdd10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2458009d90 tx=0x7f245802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cad5944ded1168 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:12.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.924+0000 7f2467e00640 1 -- 172.21.15.49:0/3688425425 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458039070 con 0x7f246815b6d0 2024-04-15T15:19:12.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.924+0000 7f2467e00640 1 -- 172.21.15.49:0/3688425425 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f245802cbe0 con 0x7f246815b6d0 2024-04-15T15:19:12.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.924+0000 7f2467e00640 1 -- 172.21.15.49:0/3688425425 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458034440 con 0x7f246815b6d0 2024-04-15T15:19:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.924+0000 7f2470200640 1 -- 172.21.15.49:0/3688425425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f246815b6d0 msgr2=0x7f24680cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.924+0000 7f2470200640 1 --2- 172.21.15.49:0/3688425425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f246815b6d0 0x7f24680cdd10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2458009d90 tx=0x7f245802be30 comp rx=0 tx=0).stop 2024-04-15T15:19:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.925+0000 7f2470200640 1 -- 172.21.15.49:0/3688425425 shutdown_connections 2024-04-15T15:19:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.925+0000 7f2470200640 1 --2- 172.21.15.49:0/3688425425 >> v2:172.21.15.107:3300/0 conn(0x7f2468070210 0x7f24680b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.925+0000 7f2470200640 1 --2- 172.21.15.49:0/3688425425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f246815b6d0 0x7f24680cdd10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.925+0000 7f2470200640 1 -- 172.21.15.49:0/3688425425 >> 172.21.15.49:0/3688425425 conn(0x7f24680b27b0 msgr2=0x7f24680b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.925+0000 7f2470200640 1 -- 172.21.15.49:0/3688425425 shutdown_connections 2024-04-15T15:19:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.925+0000 7f2470200640 1 -- 172.21.15.49:0/3688425425 wait complete. 2024-04-15T15:19:12.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.926+0000 7f2470200640 1 Processor -- start 2024-04-15T15:19:12.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.927+0000 7f2470200640 1 -- start start 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.927+0000 7f2470200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2468070210 0x7f24681480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f2470200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680ce250 0x7f2468149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f2470200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246815b6d0 0x7f2468149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f2470200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f246815cac0 con 0x7f2468070210 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f2470200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f246806f420 con 0x7f24680ce250 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f2470200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f246815cc40 con 0x7f246815b6d0 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246d800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2468070210 0x7f24681480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246d800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2468070210 0x7f24681480d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50062/0 (socket says 172.21.15.49:50062) 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246d800640 1 -- 172.21.15.49:0/1202144621 learned_addr learned my addr 172.21.15.49:0/1202144621 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246ce00640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680ce250 0x7f2468149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246d800640 1 -- 172.21.15.49:0/1202144621 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246815b6d0 msgr2=0x7f2468149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:12.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246e200640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246815b6d0 0x7f2468149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:12.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246d800640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246815b6d0 0x7f2468149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:12.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246d800640 1 -- 172.21.15.49:0/1202144621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680ce250 msgr2=0x7f2468149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:12.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246d800640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680ce250 0x7f2468149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:12.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246d800640 1 -- 172.21.15.49:0/1202144621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246814c5a0 con 0x7f2468070210 2024-04-15T15:19:12.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.928+0000 7f246e200640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246815b6d0 0x7f2468149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.929+0000 7f246d800640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2468070210 0x7f24681480d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2458037040 tx=0x7f2458004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.929+0000 7f2465600640 1 -- 172.21.15.49:0/1202144621 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458044020 con 0x7f2468070210 2024-04-15T15:19:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.929+0000 7f2465600640 1 -- 172.21.15.49:0/1202144621 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f245803f670 con 0x7f2468070210 2024-04-15T15:19:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.929+0000 7f2470200640 1 -- 172.21.15.49:0/1202144621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246814c880 con 0x7f2468070210 2024-04-15T15:19:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.929+0000 7f2465600640 1 -- 172.21.15.49:0/1202144621 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458039040 con 0x7f2468070210 2024-04-15T15:19:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.929+0000 7f2470200640 1 -- 172.21.15.49:0/1202144621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246814ce40 con 0x7f2468070210 2024-04-15T15:19:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.930+0000 7f2470200640 1 -- 172.21.15.49:0/1202144621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2468068c80 con 0x7f2468070210 2024-04-15T15:19:12.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.931+0000 7f2465600640 1 -- 172.21.15.49:0/1202144621 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2458047020 con 0x7f2468070210 2024-04-15T15:19:12.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.933+0000 7f2465600640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f244806c8f0 0x7f244806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:12.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.933+0000 7f2465600640 1 -- 172.21.15.49:0/1202144621 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f24580ae860 con 0x7f2468070210 2024-04-15T15:19:12.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.933+0000 7f246ce00640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f244806c8f0 0x7f244806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:12.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.934+0000 7f246ce00640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f244806c8f0 0x7f244806edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24680b1330 tx=0x7f2454008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:12.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:12.938+0000 7f2465600640 1 -- 172.21.15.49:0/1202144621 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f245804e030 con 0x7f2468070210 2024-04-15T15:19:13.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.183+0000 7f2470200640 1 -- 172.21.15.49:0/1202144621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f24680d0560 con 0x7f2468070210 2024-04-15T15:19:13.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.184+0000 7f2465600640 1 -- 172.21.15.49:0/1202144621 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f245807cbf0 con 0x7f2468070210 2024-04-15T15:19:13.187 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:19:13.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 -- 172.21.15.49:0/1202144621 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f244806c8f0 msgr2=0x7f244806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f244806c8f0 0x7f244806edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24680b1330 tx=0x7f2454008040 comp rx=0 tx=0).stop 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 -- 172.21.15.49:0/1202144621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2468070210 msgr2=0x7f24681480d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2468070210 0x7f24681480d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2458037040 tx=0x7f2458004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 -- 172.21.15.49:0/1202144621 shutdown_connections 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f244806c8f0 0x7f244806edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246815b6d0 0x7f2468149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680ce250 0x7f2468149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 --2- 172.21.15.49:0/1202144621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2468070210 0x7f24681480d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 -- 172.21.15.49:0/1202144621 >> 172.21.15.49:0/1202144621 conn(0x7f24680b27b0 msgr2=0x7f24680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 -- 172.21.15.49:0/1202144621 shutdown_connections 2024-04-15T15:19:13.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:13.187+0000 7f2470200640 1 -- 172.21.15.49:0/1202144621 wait complete. 2024-04-15T15:19:13.211 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:19:13.211 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:19:13.211 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-04-15T15:19:13.211 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-15T15:19:13.211 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:13.211 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:19:13.211 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:19:13.211 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:19:13.211 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:19:13.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:13.212 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-04-15T15:19:13.270 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:13.271 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:19:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:19:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42568sec preferred_lft 42568sec 2024-04-15T15:19:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-04-15T15:19:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:13.273 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-15T15:19:13.273 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:13.273 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:19:13.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-04-15T15:19:13.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-04-15T15:19:13.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-04-15T15:19:13.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-15T15:19:13.273 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:19:13.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:13.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:13.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:13.899 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:19:13.974 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-15T15:19:13.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:13.979 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-04-15T15:19:14.045 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-04-15T15:19:14.045 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:19:14.045 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:19:14.045 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:19:14.046 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:19:14.046 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:14.046 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:19:14.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:19:14.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:19:14.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:19:14.046 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:19:14.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:14.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:14.806 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-15T15:19:14.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:14.857 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-04-15T15:19:14.907 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:14.907 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:19:14.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:14.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-15T15:19:14.907 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:19:15.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:15.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:15.111 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-15T15:19:15.111 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:15.112 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:19:15.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-15T15:19:15.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-15T15:19:15.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-15T15:19:15.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-15T15:19:15.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-15T15:19:15.112 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:19:15.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:15.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:15.445 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:15.445 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:19:15.445 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-04-15T15:19:15.445 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-04-15T15:19:15.445 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:19:15.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:15.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:15.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:15.666 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:15.709 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:19:15.710 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:15.710 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:15.801 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:19:15.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:15.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:19:15.928 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:19:15.928 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:19:15.928 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:19:16.615 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:19:16.615 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:19:16.615 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:19:16.615 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:19:16.615 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:19:16.648 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:19:16.648 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:19:16.648 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:19:16.648 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:19:16.648 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:19:16.648 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790" 2024-04-15T15:19:16.648 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790:/ 2024-04-15T15:19:16.649 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c" will pass to kernel 2024-04-15T15:19:16.651 INFO:teuthology.orchestra.run.smithi107.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-15T15:19:16.651 INFO:teuthology.orchestra.run.smithi107.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:19:16.651 INFO:teuthology.orchestra.run.smithi107.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:19:16.651 INFO:teuthology.orchestra.run.smithi107.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:19:16.651 INFO:teuthology.orchestra.run.smithi107.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:19:16.656 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:19:16.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:16.656 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:19:16.744 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:16.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:16.863 INFO:teuthology.orchestra.run.smithi107.stdout:client4576 2024-04-15T15:19:16.864 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:19:16.864 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:16.914 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:17.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:17.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:17.163 INFO:teuthology.orchestra.run.smithi107.stdout:client4576 2024-04-15T15:19:17.163 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:17.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:17.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:17.289 INFO:teuthology.orchestra.run.smithi107.stdout:client4576 2024-04-15T15:19:17.289 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c.client4576/status') 2024-04-15T15:19:17.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:17.433 INFO:teuthology.orchestra.run.smithi107.stdout:instance: client.4576 (3)192.168.0.1:0/1703156645 2024-04-15T15:19:17.433 INFO:teuthology.orchestra.run.smithi107.stdout:blocklisted: false 2024-04-15T15:19:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records in 2024-04-15T15:19:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records out 2024-04-15T15:19:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:69 bytes copied, 6.4909e-05 s, 1.1 MB/s 2024-04-15T15:19:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:17.446 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-15T15:19:17.446 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:19:17.446 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:17.446 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:19:17.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.690+0000 7fe33f600640 1 Processor -- start 2024-04-15T15:19:17.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.690+0000 7fe33f600640 1 -- start start 2024-04-15T15:19:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.691+0000 7fe33f600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe3380d1300 0x7fe3380d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.691+0000 7fe33f600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b3470 0x7fe3380b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.691+0000 7fe33f600640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3380d2d90 con 0x7fe33815eac0 2024-04-15T15:19:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.691+0000 7fe33f600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fe3380d1cd0 con 0x7fe3380d1300 2024-04-15T15:19:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.691+0000 7fe33f600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe3380d3070 con 0x7fe3380b3470 2024-04-15T15:19:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.692+0000 7fe33cc00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe33815eac0 0x7fe3380d3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41768/0 (socket says 172.21.15.49:41768) 2024-04-15T15:19:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.692+0000 7fe33cc00640 1 -- 172.21.15.49:0/1328918753 learned_addr learned my addr 172.21.15.49:0/1328918753 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:17.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.692+0000 7fe337e00640 1 --2- 172.21.15.49:0/1328918753 >> v2:172.21.15.107:3300/0 conn(0x7fe3380d1300 0x7fe3380d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.692+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2040326570 0 0) 0x7fe3380d2d90 con 0x7fe33815eac0 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.692+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe31c003520 con 0x7fe33815eac0 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.692+0000 7fe33d600640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b3470 0x7fe3380b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.692+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2063306466 0 0) 0x7fe31c003520 con 0x7fe33815eac0 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.692+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b3470 msgr2=0x7fe3380b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.692+0000 7fe337400640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b3470 0x7fe3380b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 >> v2:172.21.15.107:3300/0 conn(0x7fe3380d1300 msgr2=0x7fe3380d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 --2- 172.21.15.49:0/1328918753 >> v2:172.21.15.107:3300/0 conn(0x7fe3380d1300 0x7fe3380d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3380d3350 con 0x7fe33815eac0 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fe320001670 con 0x7fe33815eac0 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 >> v1:172.21.15.49:6789/0 conn(0x7fe33815eac0 legacy=0x7fe3380d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b3470 0x7fe31c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 0x7fe31c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe31c004d40 0x7fe31c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3380d1cd0 con 0x7fe31c003f80 2024-04-15T15:19:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3380d3070 con 0x7fe31c004d40 2024-04-15T15:19:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe31c00ba70 con 0x7fe3380b3470 2024-04-15T15:19:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337e00640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 0x7fe31c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe33cc00640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b3470 0x7fe31c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe33d600640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe31c004d40 0x7fe31c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337e00640 1 -- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b3470 msgr2=0x7fe31c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.693+0000 7fe337e00640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b3470 0x7fe31c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.694+0000 7fe337e00640 1 -- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe31c004d40 msgr2=0x7fe31c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.694+0000 7fe337e00640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe31c004d40 0x7fe31c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.694+0000 7fe337e00640 1 -- 172.21.15.49:0/1328918753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31c003520 con 0x7fe31c003f80 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.694+0000 7fe33d600640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe31c004d40 0x7fe31c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.695+0000 7fe337e00640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 0x7fe31c004800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe328009fc0 tx=0x7fe31c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6292567fe091ecb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.695+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328039070 con 0x7fe31c003f80 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.695+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32802cc60 con 0x7fe31c003f80 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.695+0000 7fe337400640 1 -- 172.21.15.49:0/1328918753 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328009c10 con 0x7fe31c003f80 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.696+0000 7fe33f600640 1 -- 172.21.15.49:0/1328918753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 msgr2=0x7fe31c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.696+0000 7fe33f600640 1 --2- 172.21.15.49:0/1328918753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 0x7fe31c004800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe328009fc0 tx=0x7fe31c005830 comp rx=0 tx=0).stop 2024-04-15T15:19:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.696+0000 7fe33d600640 1 -- 172.21.15.49:0/1328918753 reap_dead start 2024-04-15T15:19:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.696+0000 7fe33f600640 1 -- 172.21.15.49:0/1328918753 shutdown_connections 2024-04-15T15:19:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.696+0000 7fe33f600640 1 -- 172.21.15.49:0/1328918753 >> 172.21.15.49:0/1328918753 conn(0x7fe338073690 msgr2=0x7fe33806cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.696+0000 7fe33f600640 1 -- 172.21.15.49:0/1328918753 shutdown_connections 2024-04-15T15:19:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.696+0000 7fe33f600640 1 -- 172.21.15.49:0/1328918753 wait complete. 2024-04-15T15:19:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.698+0000 7fe33f600640 1 Processor -- start 2024-04-15T15:19:17.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.698+0000 7fe33f600640 1 -- start start 2024-04-15T15:19:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.699+0000 7fe33f600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 0x7fe33814ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.699+0000 7fe33f600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b5d40 0x7fe3380b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.699+0000 7fe33f600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3380cba10 0x7fe3380cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.699+0000 7fe33f600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe33815eea0 con 0x7fe31c003f80 2024-04-15T15:19:17.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.699+0000 7fe33f600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3380d3910 con 0x7fe3380cba10 2024-04-15T15:19:17.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.699+0000 7fe33f600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe3380d4590 con 0x7fe3380b5d40 2024-04-15T15:19:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.699+0000 7fe33cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 0x7fe33814ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.699+0000 7fe33cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 0x7fe33814ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33928/0 (socket says 172.21.15.49:33928) 2024-04-15T15:19:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.699+0000 7fe33cc00640 1 -- 172.21.15.49:0/4290575371 learned_addr learned my addr 172.21.15.49:0/4290575371 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe337e00640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b5d40 0x7fe3380b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe33d600640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3380cba10 0x7fe3380cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe33cc00640 1 -- 172.21.15.49:0/4290575371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b5d40 msgr2=0x7fe3380b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe33cc00640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b5d40 0x7fe3380b61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe33cc00640 1 -- 172.21.15.49:0/4290575371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3380cba10 msgr2=0x7fe3380cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe33cc00640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3380cba10 0x7fe3380cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:17.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe33cc00640 1 -- 172.21.15.49:0/4290575371 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3380cc6d0 con 0x7fe31c003f80 2024-04-15T15:19:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe33cc00640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 0x7fe33814ce60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe32000e7b0 tx=0x7fe32000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe334c00640 1 -- 172.21.15.49:0/4290575371 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32000ee60 con 0x7fe31c003f80 2024-04-15T15:19:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe334c00640 1 -- 172.21.15.49:0/4290575371 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe320018070 con 0x7fe31c003f80 2024-04-15T15:19:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe334c00640 1 -- 172.21.15.49:0/4290575371 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32000cab0 con 0x7fe31c003f80 2024-04-15T15:19:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe33f600640 1 -- 172.21.15.49:0/4290575371 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3380cc9b0 con 0x7fe31c003f80 2024-04-15T15:19:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.700+0000 7fe33f600640 1 -- 172.21.15.49:0/4290575371 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3380ccf70 con 0x7fe31c003f80 2024-04-15T15:19:17.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.702+0000 7fe33f600640 1 -- 172.21.15.49:0/4290575371 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3380d29f0 con 0x7fe31c003f80 2024-04-15T15:19:17.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.703+0000 7fe334c00640 1 -- 172.21.15.49:0/4290575371 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe32000cc50 con 0x7fe31c003f80 2024-04-15T15:19:17.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.704+0000 7fe334c00640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe31406c8f0 0x7fe31406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:17.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.704+0000 7fe334c00640 1 -- 172.21.15.49:0/4290575371 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe32008b9a0 con 0x7fe31c003f80 2024-04-15T15:19:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.704+0000 7fe337e00640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe31406c8f0 0x7fe31406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.705+0000 7fe337e00640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe31406c8f0 0x7fe31406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe328036000 tx=0x7fe32802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:17.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.709+0000 7fe334c00640 1 -- 172.21.15.49:0/4290575371 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3380d29f0 con 0x7fe31c003f80 2024-04-15T15:19:17.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.960+0000 7fe33f600640 1 -- 172.21.15.49:0/4290575371 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe338068c80 con 0x7fe31c003f80 2024-04-15T15:19:17.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.961+0000 7fe334c00640 1 -- 172.21.15.49:0/4290575371 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fe320059d30 con 0x7fe31c003f80 2024-04-15T15:19:17.963 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:19:17.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 -- 172.21.15.49:0/4290575371 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe31406c8f0 msgr2=0x7fe31406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:17.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe31406c8f0 0x7fe31406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe328036000 tx=0x7fe32802e000 comp rx=0 tx=0).stop 2024-04-15T15:19:17.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 -- 172.21.15.49:0/4290575371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 msgr2=0x7fe33814ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:17.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 0x7fe33814ce60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe32000e7b0 tx=0x7fe32000ec70 comp rx=0 tx=0).stop 2024-04-15T15:19:17.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 -- 172.21.15.49:0/4290575371 shutdown_connections 2024-04-15T15:19:17.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe31406c8f0 0x7fe31406edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:17.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3380b5d40 0x7fe3380b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:17.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3380cba10 0x7fe3380cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:17.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 --2- 172.21.15.49:0/4290575371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c003f80 0x7fe33814ce60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:17.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 -- 172.21.15.49:0/4290575371 >> 172.21.15.49:0/4290575371 conn(0x7fe338073690 msgr2=0x7fe33815f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:17.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 -- 172.21.15.49:0/4290575371 shutdown_connections 2024-04-15T15:19:17.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:17.964+0000 7fe33f600640 1 -- 172.21.15.49:0/4290575371 wait complete. 2024-04-15T15:19:17.987 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:19:17.988 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:19:17.988 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-04-15T15:19:17.988 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-15T15:19:17.988 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:17.988 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:19:17.988 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:19:17.988 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:19:17.988 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:19:17.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:17.988 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-04-15T15:19:18.046 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:19:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42564sec preferred_lft 42564sec 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether fe:39:d9:9f:a9:b1 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::26:49ff:feaf:7d0d/64 scope link 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether fe:39:d9:9f:a9:b1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::fc39:d9ff:fe9f:a9b1/64 scope link 2024-04-15T15:19:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:18.050 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:19:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-15T15:19:18.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:18.143 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-04-15T15:19:18.191 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-04-15T15:19:18.191 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:19:18.191 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:19:18.191 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:19:18.191 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:19:18.192 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:18.192 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:19:18.192 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:19:18.192 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:19:18.192 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:19:18.192 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:19:18.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:18.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:18.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:18.424 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-15T15:19:18.468 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:19:18.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:18.470 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-04-15T15:19:18.565 INFO:teuthology.orchestra.run.smithi107.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-15T15:19:18.566 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:18.566 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:19:18.566 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:19:18.566 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-15T15:19:18.566 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:19:18.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:18.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:18.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:18.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-15T15:19:18.860 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:18.860 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:18.861 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:18.861 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.861 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:18.861 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.861 INFO:teuthology.orchestra.run.smithi107.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:18.861 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 02:cb:04:38:60:2f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:19:18.861 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:19:18.861 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.862 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::cb:4ff:fe38:602f/64 scope link 2024-04-15T15:19:18.862 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:18.865 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-15T15:19:18.950 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:19:18.950 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:19:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:19:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-15T15:19:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.951 INFO:teuthology.orchestra.run.smithi107.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:19:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 02:cb:04:38:60:2f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:19:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:19:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::cb:4ff:fe38:602f/64 scope link 2024-04-15T15:19:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:19:18.955 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-15T15:19:18.955 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:18.955 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:19:18.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-15T15:19:18.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-15T15:19:18.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-15T15:19:18.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-15T15:19:18.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-15T15:19:18.955 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:19:19.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:19.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:19.277 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:19:19.278 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:19:19.278 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.1 up 2024-04-15T15:19:19.278 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.1 master ceph-brx 2024-04-15T15:19:19.278 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:19:19.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:19.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:19.500 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:19.500 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:19.544 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:19:19.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:19:19.545 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:19.638 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:19:19.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:19.639 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:19:19.777 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:19:19.777 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:19:19.777 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:19:19.798 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:19:19.798 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:19:19.798 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:19:19.798 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:19:19.798 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:19:19.826 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:19:19.826 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:19:19.826 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:19:19.826 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:19:19.826 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:19:19.826 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790" 2024-04-15T15:19:19.826 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790:/ 2024-04-15T15:19:19.826 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c" will pass to kernel 2024-04-15T15:19:19.828 INFO:teuthology.orchestra.run.smithi107.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-15T15:19:19.828 INFO:teuthology.orchestra.run.smithi107.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:19:19.828 INFO:teuthology.orchestra.run.smithi107.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:19:19.828 INFO:teuthology.orchestra.run.smithi107.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:19:19.828 INFO:teuthology.orchestra.run.smithi107.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:19:19.835 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:19:19.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:19.835 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:19:19.917 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:20.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:20.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:20.038 INFO:teuthology.orchestra.run.smithi107.stdout:client4671 2024-04-15T15:19:20.038 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:19:20.038 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:20.086 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:20.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:20.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:20.251 INFO:teuthology.orchestra.run.smithi107.stdout:client4671 2024-04-15T15:19:20.252 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:19:20.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:19:20.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:20.367 INFO:teuthology.orchestra.run.smithi107.stdout:client4671 2024-04-15T15:19:20.368 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c.client4671/status') 2024-04-15T15:19:20.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:19:20.509 INFO:teuthology.orchestra.run.smithi107.stdout:instance: client.4671 (3)172.21.15.107:0/1518266838 2024-04-15T15:19:20.509 INFO:teuthology.orchestra.run.smithi107.stdout:blocklisted: false 2024-04-15T15:19:20.509 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records in 2024-04-15T15:19:20.509 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records out 2024-04-15T15:19:20.509 INFO:teuthology.orchestra.run.smithi107.stderr:71 bytes copied, 7.7189e-05 s, 920 kB/s 2024-04-15T15:19:20.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:19:20.522 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-15T15:19:20.540 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:20.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.777+0000 7f355b400640 1 Processor -- start 2024-04-15T15:19:20.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.777+0000 7f355b400640 1 -- start start 2024-04-15T15:19:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f355b400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35541461f0 0x7f35540ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f355b400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f35540cedf0 0x7f35540b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f355b400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f35540b3e10 0x7f35540cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f355b400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f35540a9640 con 0x7f35540b3e10 2024-04-15T15:19:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f355b400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f35540cf250 con 0x7f35540cedf0 2024-04-15T15:19:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f355b400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f35540b4be0 con 0x7f35541461f0 2024-04-15T15:19:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f3559400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f35540b3e10 0x7f35540cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f3559400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f35540b3e10 0x7f35540cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33936/0 (socket says 172.21.15.49:33936) 2024-04-15T15:19:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f3559400640 1 -- 172.21.15.49:0/302370678 learned_addr learned my addr 172.21.15.49:0/302370678 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:20.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f3553e00640 1 --2- 172.21.15.49:0/302370678 >> v2:172.21.15.107:3300/0 conn(0x7f35540cedf0 0x7f35540b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:20.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.779+0000 7f3558a00640 1 --2- 172.21.15.49:0/302370678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35541461f0 0x7f35540ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:20.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.780+0000 7f3559400640 1 -- 172.21.15.49:0/302370678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35541461f0 msgr2=0x7f35540ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:20.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.780+0000 7f3559400640 1 --2- 172.21.15.49:0/302370678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35541461f0 0x7f35540ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.780+0000 7f3559400640 1 -- 172.21.15.49:0/302370678 >> v2:172.21.15.107:3300/0 conn(0x7f35540cedf0 msgr2=0x7f35540b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:20.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.780+0000 7f3559400640 1 --2- 172.21.15.49:0/302370678 >> v2:172.21.15.107:3300/0 conn(0x7f35540cedf0 0x7f35540b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.780+0000 7f3559400640 1 -- 172.21.15.49:0/302370678 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35540b4620 con 0x7f35540b3e10 2024-04-15T15:19:20.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.781+0000 7f3559400640 1 --2- 172.21.15.49:0/302370678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 0x7f35540cd2b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f354c009e50 tx=0x7f354c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57450ecd90f3b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:20.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.782+0000 7f3553400640 1 -- 172.21.15.49:0/302370678 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c02ca70 con 0x7f35540b3e10 2024-04-15T15:19:20.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.782+0000 7f3553400640 1 -- 172.21.15.49:0/302370678 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354c02cc10 con 0x7f35540b3e10 2024-04-15T15:19:20.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.782+0000 7f3553400640 1 -- 172.21.15.49:0/302370678 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c0343f0 con 0x7f35540b3e10 2024-04-15T15:19:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.783+0000 7f355b400640 1 -- 172.21.15.49:0/302370678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 msgr2=0x7f35540cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.783+0000 7f355b400640 1 --2- 172.21.15.49:0/302370678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 0x7f35540cd2b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f354c009e50 tx=0x7f354c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.783+0000 7f355b400640 1 -- 172.21.15.49:0/302370678 shutdown_connections 2024-04-15T15:19:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.783+0000 7f355b400640 1 --2- 172.21.15.49:0/302370678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 0x7f35540cd2b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.783+0000 7f355b400640 1 --2- 172.21.15.49:0/302370678 >> v2:172.21.15.107:3300/0 conn(0x7f35540cedf0 0x7f35540b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.783+0000 7f355b400640 1 --2- 172.21.15.49:0/302370678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35541461f0 0x7f35540ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.783+0000 7f355b400640 1 -- 172.21.15.49:0/302370678 >> 172.21.15.49:0/302370678 conn(0x7f35540b27b0 msgr2=0x7f35540b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.783+0000 7f355b400640 1 -- 172.21.15.49:0/302370678 shutdown_connections 2024-04-15T15:19:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.783+0000 7f355b400640 1 -- 172.21.15.49:0/302370678 wait complete. 2024-04-15T15:19:20.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.785+0000 7f355b400640 1 Processor -- start 2024-04-15T15:19:20.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.786+0000 7f355b400640 1 -- start start 2024-04-15T15:19:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.787+0000 7f355b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 0x7f355415c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.787+0000 7f355b400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35540cedf0 0x7f3554143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.787+0000 7f355b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35541461f0 0x7f35541435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.787+0000 7f355b400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f355415d290 con 0x7f35540b3e10 2024-04-15T15:19:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.787+0000 7f355b400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f355415cfb0 con 0x7f35541461f0 2024-04-15T15:19:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.787+0000 7f355b400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f35541465d0 con 0x7f35540cedf0 2024-04-15T15:19:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.787+0000 7f3558a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 0x7f355415c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.787+0000 7f3558a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 0x7f355415c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33952/0 (socket says 172.21.15.49:33952) 2024-04-15T15:19:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.787+0000 7f3558a00640 1 -- 172.21.15.49:0/1609864247 learned_addr learned my addr 172.21.15.49:0/1609864247 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:20.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3559400640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35541461f0 0x7f35541435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:20.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3553e00640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35540cedf0 0x7f3554143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:20.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3558a00640 1 -- 172.21.15.49:0/1609864247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35540cedf0 msgr2=0x7f3554143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3558a00640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35540cedf0 0x7f3554143080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3558a00640 1 -- 172.21.15.49:0/1609864247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35541461f0 msgr2=0x7f35541435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3558a00640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35541461f0 0x7f35541435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3558a00640 1 -- 172.21.15.49:0/1609864247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35540add80 con 0x7f35540b3e10 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3553e00640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35540cedf0 0x7f3554143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3559400640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35541461f0 0x7f35541435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3558a00640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 0x7f355415c1f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f354800d620 tx=0x7f354800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.788+0000 7f3550c00640 1 -- 172.21.15.49:0/1609864247 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354800b620 con 0x7f35540b3e10 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.789+0000 7f3550c00640 1 -- 172.21.15.49:0/1609864247 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3548019070 con 0x7f35540b3e10 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.789+0000 7f355b400640 1 -- 172.21.15.49:0/1609864247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35540ae060 con 0x7f35540b3e10 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.789+0000 7f3550c00640 1 -- 172.21.15.49:0/1609864247 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354800ba00 con 0x7f35540b3e10 2024-04-15T15:19:20.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.789+0000 7f355b400640 1 -- 172.21.15.49:0/1609864247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35540ae620 con 0x7f35540b3e10 2024-04-15T15:19:20.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.790+0000 7f355b400640 1 -- 172.21.15.49:0/1609864247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3554068c80 con 0x7f35540b3e10 2024-04-15T15:19:20.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.791+0000 7f3550c00640 1 -- 172.21.15.49:0/1609864247 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f354800bba0 con 0x7f35540b3e10 2024-04-15T15:19:20.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.792+0000 7f3550c00640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f353406c8f0 0x7f353406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:20.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.792+0000 7f3550c00640 1 -- 172.21.15.49:0/1609864247 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f354808bfa0 con 0x7f35540b3e10 2024-04-15T15:19:20.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.792+0000 7f3553e00640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f353406c8f0 0x7f353406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:20.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.793+0000 7f3553e00640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f353406c8f0 0x7f353406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35540b51a0 tx=0x7f3540008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:20.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:20.797+0000 7f3550c00640 1 -- 172.21.15.49:0/1609864247 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f354805a370 con 0x7f35540b3e10 2024-04-15T15:19:21.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.004+0000 7f355b400640 1 -- 172.21.15.49:0/1609864247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35540cdea0 con 0x7f35540b3e10 2024-04-15T15:19:21.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.007+0000 7f3550c00640 1 -- 172.21.15.49:0/1609864247 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7f3548060030 con 0x7f35540b3e10 2024-04-15T15:19:21.009 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:21.009 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.011+0000 7f355b400640 1 -- 172.21.15.49:0/1609864247 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f353406c8f0 msgr2=0x7f353406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.011+0000 7f355b400640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f353406c8f0 0x7f353406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35540b51a0 tx=0x7f3540008040 comp rx=0 tx=0).stop 2024-04-15T15:19:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.011+0000 7f355b400640 1 -- 172.21.15.49:0/1609864247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 msgr2=0x7f355415c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.011+0000 7f355b400640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 0x7f355415c1f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f354800d620 tx=0x7f354800dae0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.011+0000 7f355b400640 1 -- 172.21.15.49:0/1609864247 shutdown_connections 2024-04-15T15:19:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.011+0000 7f355b400640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f353406c8f0 0x7f353406edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.011+0000 7f355b400640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35540cedf0 0x7f3554143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.011+0000 7f355b400640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35541461f0 0x7f35541435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.011+0000 7f355b400640 1 --2- 172.21.15.49:0/1609864247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35540b3e10 0x7f355415c1f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.011+0000 7f355b400640 1 -- 172.21.15.49:0/1609864247 >> 172.21.15.49:0/1609864247 conn(0x7f35540b27b0 msgr2=0x7f355406e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.012+0000 7f355b400640 1 -- 172.21.15.49:0/1609864247 shutdown_connections 2024-04-15T15:19:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.012+0000 7f355b400640 1 -- 172.21.15.49:0/1609864247 wait complete. 2024-04-15T15:19:21.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:21.033 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:21.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.255+0000 7fcd1be00640 1 Processor -- start 2024-04-15T15:19:21.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.256+0000 7fcd1be00640 1 -- start start 2024-04-15T15:19:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.257+0000 7fcd1be00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c15d510 0x7fcd1c06ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.257+0000 7fcd1be00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcd1c15d8f0 0x7fcd1c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.257+0000 7fcd1be00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1c0cecd0 con 0x7fcd1c0ce850 2024-04-15T15:19:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.258+0000 7fcd1be00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1c0cefb0 con 0x7fcd1c15d8f0 2024-04-15T15:19:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.258+0000 7fcd1be00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c06dfa0 con 0x7fcd1c15d510 2024-04-15T15:19:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.258+0000 7fcd1b400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcd1c0ce850 0x7fcd1c0b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41784/0 (socket says 172.21.15.49:41784) 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.258+0000 7fcd1b400640 1 -- 172.21.15.49:0/173841549 learned_addr learned my addr 172.21.15.49:0/173841549 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.258+0000 7fcd1aa00640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c15d510 0x7fcd1c06ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.258+0000 7fcd1a000640 1 --2- 172.21.15.49:0/173841549 >> v2:172.21.15.107:3300/0 conn(0x7fcd1c15d8f0 0x7fcd1c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.258+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4017177019 0 0) 0x7fcd1c0cecd0 con 0x7fcd1c0ce850 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd00003520 con 0x7fcd1c0ce850 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3111474274 0 0) 0x7fcd00003520 con 0x7fcd1c0ce850 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c15d510 msgr2=0x7fcd1c06ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c15d510 0x7fcd1c06ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 >> v2:172.21.15.107:3300/0 conn(0x7fcd1c15d8f0 msgr2=0x7fcd1c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 --2- 172.21.15.49:0/173841549 >> v2:172.21.15.107:3300/0 conn(0x7fcd1c15d8f0 0x7fcd1c0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c06e280 con 0x7fcd1c0ce850 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fcd140019f0 con 0x7fcd1c0ce850 2024-04-15T15:19:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 >> v1:172.21.15.49:6789/0 conn(0x7fcd1c0ce850 legacy=0x7fcd1c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd000039b0 0x7fcd00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd00004800 0x7fcd00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c15d510 0x7fcd000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c0cefb0 con 0x7fcd00004800 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c06dfa0 con 0x7fcd000039b0 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.259+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0000fdd0 con 0x7fcd1c15d510 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.260+0000 7fcd1a000640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd00004800 0x7fcd00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.260+0000 7fcd1aa00640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd000039b0 0x7fcd00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.260+0000 7fcd1b400640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c15d510 0x7fcd000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.260+0000 7fcd1a000640 1 -- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c15d510 msgr2=0x7fcd000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.260+0000 7fcd1a000640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c15d510 0x7fcd000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.260+0000 7fcd1a000640 1 -- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd000039b0 msgr2=0x7fcd00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.260+0000 7fcd1a000640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd000039b0 0x7fcd00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.260+0000 7fcd1a000640 1 -- 172.21.15.49:0/173841549 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd00003520 con 0x7fcd00004800 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.261+0000 7fcd1a000640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd00004800 0x7fcd00004c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009ac0 tx=0x7fcd00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccb40ee14fb921a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.262+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08039070 con 0x7fcd00004800 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.262+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0802c6d0 con 0x7fcd00004800 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.262+0000 7fcd19600640 1 -- 172.21.15.49:0/173841549 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08009690 con 0x7fcd00004800 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.262+0000 7fcd1be00640 1 -- 172.21.15.49:0/173841549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd00004800 msgr2=0x7fcd00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.262+0000 7fcd1be00640 1 --2- 172.21.15.49:0/173841549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd00004800 0x7fcd00004c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009ac0 tx=0x7fcd00005830 comp rx=0 tx=0).stop 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.262+0000 7fcd1b400640 1 -- 172.21.15.49:0/173841549 reap_dead start 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.262+0000 7fcd1be00640 1 -- 172.21.15.49:0/173841549 shutdown_connections 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.262+0000 7fcd1be00640 1 -- 172.21.15.49:0/173841549 >> 172.21.15.49:0/173841549 conn(0x7fcd1c0b23a0 msgr2=0x7fcd1c0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.262+0000 7fcd1be00640 1 -- 172.21.15.49:0/173841549 shutdown_connections 2024-04-15T15:19:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.262+0000 7fcd1be00640 1 -- 172.21.15.49:0/173841549 wait complete. 2024-04-15T15:19:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.264+0000 7fcd1be00640 1 Processor -- start 2024-04-15T15:19:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.264+0000 7fcd1be00640 1 -- start start 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.265+0000 7fcd1be00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd00004800 0x7fcd1c146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.265+0000 7fcd1be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c155880 0x7fcd1c1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.265+0000 7fcd1be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd1c155d00 0x7fcd1c157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1be00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c0b6240 con 0x7fcd1c155d00 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c0b60c0 con 0x7fcd1c155880 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1be00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c06ea80 con 0x7fcd00004800 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd1c155d00 0x7fcd1c157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd1c155d00 0x7fcd1c157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33966/0 (socket says 172.21.15.49:33966) 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1b400640 1 -- 172.21.15.49:0/1761279055 learned_addr learned my addr 172.21.15.49:0/1761279055 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1a000640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c155880 0x7fcd1c1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1aa00640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd00004800 0x7fcd1c146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1b400640 1 -- 172.21.15.49:0/1761279055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd00004800 msgr2=0x7fcd1c146eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1b400640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd00004800 0x7fcd1c146eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1b400640 1 -- 172.21.15.49:0/1761279055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c155880 msgr2=0x7fcd1c1574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1b400640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c155880 0x7fcd1c1574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.266+0000 7fcd1b400640 1 -- 172.21.15.49:0/1761279055 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c157290 con 0x7fcd1c155d00 2024-04-15T15:19:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.267+0000 7fcd1b400640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd1c155d00 0x7fcd1c157a30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c071d30 tx=0x7fcd1400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.267+0000 7fcd12a00640 1 -- 172.21.15.49:0/1761279055 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1400ec10 con 0x7fcd1c155d00 2024-04-15T15:19:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.267+0000 7fcd12a00640 1 -- 172.21.15.49:0/1761279055 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1400edb0 con 0x7fcd1c155d00 2024-04-15T15:19:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.267+0000 7fcd12a00640 1 -- 172.21.15.49:0/1761279055 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14014670 con 0x7fcd1c155d00 2024-04-15T15:19:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.267+0000 7fcd1be00640 1 -- 172.21.15.49:0/1761279055 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1c1561f0 con 0x7fcd1c155d00 2024-04-15T15:19:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.267+0000 7fcd1be00640 1 -- 172.21.15.49:0/1761279055 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1c1567b0 con 0x7fcd1c155d00 2024-04-15T15:19:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.268+0000 7fcd1be00640 1 -- 172.21.15.49:0/1761279055 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1c0755c0 con 0x7fcd1c155d00 2024-04-15T15:19:21.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.269+0000 7fcd12a00640 1 -- 172.21.15.49:0/1761279055 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd14014810 con 0x7fcd1c155d00 2024-04-15T15:19:21.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.270+0000 7fcd12a00640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fccfc06c920 0x7fccfc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.270+0000 7fcd12a00640 1 -- 172.21.15.49:0/1761279055 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcd1408b8a0 con 0x7fcd1c155d00 2024-04-15T15:19:21.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.271+0000 7fcd1aa00640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fccfc06c920 0x7fccfc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.271+0000 7fcd1aa00640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fccfc06c920 0x7fccfc06ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0af530 tx=0x7fcd0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:21.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.275+0000 7fcd12a00640 1 -- 172.21.15.49:0/1761279055 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd14059d60 con 0x7fcd1c155d00 2024-04-15T15:19:21.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.478+0000 7fcd1be00640 1 -- 172.21.15.49:0/1761279055 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd1c15c1e0 con 0x7fcd1c155d00 2024-04-15T15:19:21.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.479+0000 7fcd12a00640 1 -- 172.21.15.49:0/1761279055 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fcd14059660 con 0x7fcd1c155d00 2024-04-15T15:19:21.480 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:21.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.481+0000 7fcd1be00640 1 -- 172.21.15.49:0/1761279055 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fccfc06c920 msgr2=0x7fccfc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.481+0000 7fcd1be00640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fccfc06c920 0x7fccfc06ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0af530 tx=0x7fcd0c008040 comp rx=0 tx=0).stop 2024-04-15T15:19:21.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.481+0000 7fcd1be00640 1 -- 172.21.15.49:0/1761279055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd1c155d00 msgr2=0x7fcd1c157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.481+0000 7fcd1be00640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd1c155d00 0x7fcd1c157a30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c071d30 tx=0x7fcd1400e530 comp rx=0 tx=0).stop 2024-04-15T15:19:21.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.482+0000 7fcd1be00640 1 -- 172.21.15.49:0/1761279055 shutdown_connections 2024-04-15T15:19:21.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.482+0000 7fcd1be00640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fccfc06c920 0x7fccfc06ede0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.482+0000 7fcd1be00640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd1c155d00 0x7fcd1c157a30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.482+0000 7fcd1be00640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c155880 0x7fcd1c1574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.482+0000 7fcd1be00640 1 --2- 172.21.15.49:0/1761279055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd00004800 0x7fcd1c146eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.482+0000 7fcd1be00640 1 -- 172.21.15.49:0/1761279055 >> 172.21.15.49:0/1761279055 conn(0x7fcd1c0b23a0 msgr2=0x7fcd1c0b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:21.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.482+0000 7fcd1be00640 1 -- 172.21.15.49:0/1761279055 shutdown_connections 2024-04-15T15:19:21.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.482+0000 7fcd1be00640 1 -- 172.21.15.49:0/1761279055 wait complete. 2024-04-15T15:19:21.504 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:21.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.737+0000 7f6c60a00640 1 Processor -- start 2024-04-15T15:19:21.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.737+0000 7f6c60a00640 1 -- start start 2024-04-15T15:19:21.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.738+0000 7f6c60a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c5c1461c0 0x7f6c5c071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.738+0000 7f6c60a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6c5c072420 0x7f6c5c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.738+0000 7f6c60a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c5c0cd2d0 con 0x7f6c5c0ce9e0 2024-04-15T15:19:21.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.738+0000 7f6c60a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f6c5c0cf1d0 con 0x7f6c5c072420 2024-04-15T15:19:21.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.738+0000 7f6c60a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5c072880 con 0x7f6c5c1461c0 2024-04-15T15:19:21.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.739+0000 7f6c59e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6c5c072420 0x7f6c5c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.739+0000 7f6c5b200640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6c5c0ce9e0 0x7f6c5c160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41792/0 (socket says 172.21.15.49:41792) 2024-04-15T15:19:21.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.739+0000 7f6c59e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6c5c072420 0x7f6c5c0ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33970/0 (socket says 172.21.15.49:33970) 2024-04-15T15:19:21.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.739+0000 7f6c5b200640 1 -- 172.21.15.49:0/3441468343 learned_addr learned my addr 172.21.15.49:0/3441468343 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:21.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.739+0000 7f6c5a800640 1 --2- 172.21.15.49:0/3441468343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c5c1461c0 0x7f6c5c071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.739+0000 7f6c59e00640 1 -- 172.21.15.49:0/3441468343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c5c1461c0 msgr2=0x7f6c5c071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:21.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.739+0000 7f6c59e00640 1 --2- 172.21.15.49:0/3441468343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c5c1461c0 0x7f6c5c071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.739+0000 7f6c59e00640 1 -- 172.21.15.49:0/3441468343 >> v1:172.21.15.49:6789/0 conn(0x7f6c5c0ce9e0 legacy=0x7f6c5c160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.739+0000 7f6c59e00640 1 -- 172.21.15.49:0/3441468343 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c0cd5b0 con 0x7f6c5c072420 2024-04-15T15:19:21.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.740+0000 7f6c59e00640 1 --2- 172.21.15.49:0/3441468343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c072420 0x7f6c5c0ce4a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c440097a0 tx=0x7f6c4402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e29076096a3da2a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:21.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.741+0000 7f6c59400640 1 -- 172.21.15.49:0/3441468343 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44035020 con 0x7f6c5c072420 2024-04-15T15:19:21.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.741+0000 7f6c59400640 1 -- 172.21.15.49:0/3441468343 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c44031810 con 0x7f6c5c072420 2024-04-15T15:19:21.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.741+0000 7f6c59400640 1 -- 172.21.15.49:0/3441468343 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44031be0 con 0x7f6c5c072420 2024-04-15T15:19:21.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.741+0000 7f6c60a00640 1 -- 172.21.15.49:0/3441468343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c072420 msgr2=0x7f6c5c0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.741+0000 7f6c60a00640 1 --2- 172.21.15.49:0/3441468343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c072420 0x7f6c5c0ce4a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c440097a0 tx=0x7f6c4402b8b0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.741+0000 7f6c60a00640 1 -- 172.21.15.49:0/3441468343 shutdown_connections 2024-04-15T15:19:21.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.741+0000 7f6c60a00640 1 --2- 172.21.15.49:0/3441468343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c072420 0x7f6c5c0ce4a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.741+0000 7f6c60a00640 1 --2- 172.21.15.49:0/3441468343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c5c1461c0 0x7f6c5c071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.741+0000 7f6c60a00640 1 -- 172.21.15.49:0/3441468343 >> 172.21.15.49:0/3441468343 conn(0x7f6c5c06b930 msgr2=0x7f6c5c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:21.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.742+0000 7f6c60a00640 1 -- 172.21.15.49:0/3441468343 shutdown_connections 2024-04-15T15:19:21.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.742+0000 7f6c60a00640 1 -- 172.21.15.49:0/3441468343 wait complete. 2024-04-15T15:19:21.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.743+0000 7f6c60a00640 1 Processor -- start 2024-04-15T15:19:21.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.743+0000 7f6c60a00640 1 -- start start 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.744+0000 7f6c60a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c5c072420 0x7f6c5c1f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.744+0000 7f6c60a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c5c0ce9e0 0x7f6c5c1f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.744+0000 7f6c60a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c1461c0 0x7f6c5c06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.744+0000 7f6c60a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c0cdb70 con 0x7f6c5c1461c0 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.744+0000 7f6c60a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c0cd080 con 0x7f6c5c072420 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.744+0000 7f6c60a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5c0cee70 con 0x7f6c5c0ce9e0 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.745+0000 7f6c5b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c1461c0 0x7f6c5c06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.745+0000 7f6c5b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c1461c0 0x7f6c5c06ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33978/0 (socket says 172.21.15.49:33978) 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.745+0000 7f6c5b200640 1 -- 172.21.15.49:0/2423372610 learned_addr learned my addr 172.21.15.49:0/2423372610 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.745+0000 7f6c59e00640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c5c0ce9e0 0x7f6c5c1f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.745+0000 7f6c5a800640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c5c072420 0x7f6c5c1f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.745+0000 7f6c5b200640 1 -- 172.21.15.49:0/2423372610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c5c0ce9e0 msgr2=0x7f6c5c1f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.745+0000 7f6c5b200640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c5c0ce9e0 0x7f6c5c1f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.745+0000 7f6c5b200640 1 -- 172.21.15.49:0/2423372610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c5c072420 msgr2=0x7f6c5c1f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.745+0000 7f6c5b200640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c5c072420 0x7f6c5c1f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.745+0000 7f6c5b200640 1 -- 172.21.15.49:0/2423372610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c1f8b80 con 0x7f6c5c1461c0 2024-04-15T15:19:21.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.746+0000 7f6c5b200640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c1461c0 0x7f6c5c06ea70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400c420 tx=0x7f6c5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:21.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.746+0000 7f6c52a00640 1 -- 172.21.15.49:0/2423372610 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54017070 con 0x7f6c5c1461c0 2024-04-15T15:19:21.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.746+0000 7f6c52a00640 1 -- 172.21.15.49:0/2423372610 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c54005ce0 con 0x7f6c5c1461c0 2024-04-15T15:19:21.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.746+0000 7f6c52a00640 1 -- 172.21.15.49:0/2423372610 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c540072d0 con 0x7f6c5c1461c0 2024-04-15T15:19:21.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.746+0000 7f6c60a00640 1 -- 172.21.15.49:0/2423372610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c1f4950 con 0x7f6c5c1461c0 2024-04-15T15:19:21.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.746+0000 7f6c60a00640 1 -- 172.21.15.49:0/2423372610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c1f4f10 con 0x7f6c5c1461c0 2024-04-15T15:19:21.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.747+0000 7f6c60a00640 1 -- 172.21.15.49:0/2423372610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c5c1386e0 con 0x7f6c5c1461c0 2024-04-15T15:19:21.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.748+0000 7f6c52a00640 1 -- 172.21.15.49:0/2423372610 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c5401a070 con 0x7f6c5c1461c0 2024-04-15T15:19:21.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.749+0000 7f6c52a00640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c3806c8f0 0x7f6c3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:21.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.749+0000 7f6c52a00640 1 -- 172.21.15.49:0/2423372610 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6c5408ae50 con 0x7f6c5c1461c0 2024-04-15T15:19:21.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.749+0000 7f6c5a800640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c3806c8f0 0x7f6c3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:21.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.750+0000 7f6c5a800640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c3806c8f0 0x7f6c3806edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c06f750 tx=0x7f6c4c0039b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:21.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.754+0000 7f6c52a00640 1 -- 172.21.15.49:0/2423372610 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c5c1386e0 con 0x7f6c5c1461c0 2024-04-15T15:19:21.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.963+0000 7f6c60a00640 1 -- 172.21.15.49:0/2423372610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6c5c0755c0 con 0x7f6c5c1461c0 2024-04-15T15:19:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.964+0000 7f6c52a00640 1 -- 172.21.15.49:0/2423372610 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f6c5c1f8b80 con 0x7f6c5c1461c0 2024-04-15T15:19:21.966 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.967+0000 7f6c60a00640 1 -- 172.21.15.49:0/2423372610 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c3806c8f0 msgr2=0x7f6c3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.967+0000 7f6c60a00640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c3806c8f0 0x7f6c3806edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c06f750 tx=0x7f6c4c0039b0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.967+0000 7f6c60a00640 1 -- 172.21.15.49:0/2423372610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c1461c0 msgr2=0x7f6c5c06ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:21.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.967+0000 7f6c60a00640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c1461c0 0x7f6c5c06ea70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400c420 tx=0x7f6c5400c8e0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.968+0000 7f6c60a00640 1 -- 172.21.15.49:0/2423372610 shutdown_connections 2024-04-15T15:19:21.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.968+0000 7f6c60a00640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c3806c8f0 0x7f6c3806edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.968+0000 7f6c60a00640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c5c0ce9e0 0x7f6c5c1f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.968+0000 7f6c60a00640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c5c1461c0 0x7f6c5c06ea70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.968+0000 7f6c60a00640 1 --2- 172.21.15.49:0/2423372610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c5c072420 0x7f6c5c1f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.968+0000 7f6c60a00640 1 -- 172.21.15.49:0/2423372610 >> 172.21.15.49:0/2423372610 conn(0x7f6c5c06b930 msgr2=0x7f6c5c0b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.968+0000 7f6c60a00640 1 -- 172.21.15.49:0/2423372610 shutdown_connections 2024-04-15T15:19:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:21.968+0000 7f6c60a00640 1 -- 172.21.15.49:0/2423372610 wait complete. 2024-04-15T15:19:21.991 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:22.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.227+0000 7f5eb7200640 1 Processor -- start 2024-04-15T15:19:22.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.227+0000 7f5eb7200640 1 -- start start 2024-04-15T15:19:22.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.228+0000 7f5eb7200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5eb8162730 0x7f5eb80b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.228+0000 7f5eb7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb80d2b50 0x7f5eb80d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.228+0000 7f5eb7200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb8162dd0 con 0x7f5eb815c680 2024-04-15T15:19:22.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.228+0000 7f5eb7200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb8162b90 con 0x7f5eb8162730 2024-04-15T15:19:22.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.228+0000 7f5eb7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb81630b0 con 0x7f5eb80d2b50 2024-04-15T15:19:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.229+0000 7f5eb5400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5eb8162730 0x7f5eb80b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.229+0000 7f5eb5e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5eb815c680 0x7f5eb815f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41796/0 (socket says 172.21.15.49:41796) 2024-04-15T15:19:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.229+0000 7f5eb5e00640 1 -- 172.21.15.49:0/3267193288 learned_addr learned my addr 172.21.15.49:0/3267193288 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.229+0000 7f5eb6800640 1 --2- 172.21.15.49:0/3267193288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb80d2b50 0x7f5eb80d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.229+0000 7f5eb5400640 1 -- 172.21.15.49:0/3267193288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb80d2b50 msgr2=0x7f5eb80d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.229+0000 7f5eb5400640 1 --2- 172.21.15.49:0/3267193288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb80d2b50 0x7f5eb80d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.229+0000 7f5eb5400640 1 -- 172.21.15.49:0/3267193288 >> v1:172.21.15.49:6789/0 conn(0x7f5eb815c680 legacy=0x7f5eb815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.229+0000 7f5eb4a00640 1 -- 172.21.15.49:0/3267193288 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45567640 0 0) 0x7f5eb8162dd0 con 0x7f5eb815c680 2024-04-15T15:19:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.230+0000 7f5eb5400640 1 -- 172.21.15.49:0/3267193288 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb8163390 con 0x7f5eb8162730 2024-04-15T15:19:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.231+0000 7f5eb5400640 1 --2- 172.21.15.49:0/3267193288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb8162730 0x7f5eb80b6c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40097a0 tx=0x7f5ea402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89edd5a29351cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.231+0000 7f5eb4a00640 1 -- 172.21.15.49:0/3267193288 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4035020 con 0x7f5eb8162730 2024-04-15T15:19:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.231+0000 7f5eb4a00640 1 -- 172.21.15.49:0/3267193288 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea4031830 con 0x7f5eb8162730 2024-04-15T15:19:22.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.231+0000 7f5eb4a00640 1 -- 172.21.15.49:0/3267193288 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4031c00 con 0x7f5eb8162730 2024-04-15T15:19:22.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.232+0000 7f5eb7200640 1 -- 172.21.15.49:0/3267193288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb8162730 msgr2=0x7f5eb80b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.232+0000 7f5eb7200640 1 --2- 172.21.15.49:0/3267193288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb8162730 0x7f5eb80b6c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40097a0 tx=0x7f5ea402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.232+0000 7f5eb7200640 1 -- 172.21.15.49:0/3267193288 shutdown_connections 2024-04-15T15:19:22.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.232+0000 7f5eb7200640 1 --2- 172.21.15.49:0/3267193288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb80d2b50 0x7f5eb80d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.232+0000 7f5eb7200640 1 --2- 172.21.15.49:0/3267193288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb8162730 0x7f5eb80b6c70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.232+0000 7f5eb7200640 1 -- 172.21.15.49:0/3267193288 >> 172.21.15.49:0/3267193288 conn(0x7f5eb8073480 msgr2=0x7f5eb8073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:22.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.232+0000 7f5eb7200640 1 -- 172.21.15.49:0/3267193288 shutdown_connections 2024-04-15T15:19:22.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.232+0000 7f5eb7200640 1 -- 172.21.15.49:0/3267193288 wait complete. 2024-04-15T15:19:22.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.234+0000 7f5eb7200640 1 Processor -- start 2024-04-15T15:19:22.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.234+0000 7f5eb7200640 1 -- start start 2024-04-15T15:19:22.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.235+0000 7f5eb7200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb80d2b50 0x7f5eb81f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.235+0000 7f5eb7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb815c680 0x7f5eb815cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.235+0000 7f5eb7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8162730 0x7f5eb8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.235+0000 7f5eb7200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb815ff60 con 0x7f5eb80d2b50 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.235+0000 7f5eb7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb815ca60 con 0x7f5eb8162730 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.235+0000 7f5eb7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb80d1770 con 0x7f5eb815c680 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.235+0000 7f5eb5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb80d2b50 0x7f5eb81f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.235+0000 7f5eb5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb80d2b50 0x7f5eb81f4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33998/0 (socket says 172.21.15.49:33998) 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.235+0000 7f5eb5e00640 1 -- 172.21.15.49:0/1591352914 learned_addr learned my addr 172.21.15.49:0/1591352914 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.235+0000 7f5eb5400640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb815c680 0x7f5eb815cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eb6800640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8162730 0x7f5eb8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eb5e00640 1 -- 172.21.15.49:0/1591352914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb815c680 msgr2=0x7f5eb815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eb5e00640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb815c680 0x7f5eb815cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eb5e00640 1 -- 172.21.15.49:0/1591352914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8162730 msgr2=0x7f5eb8155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eb5e00640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8162730 0x7f5eb8155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eb5e00640 1 -- 172.21.15.49:0/1591352914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb81562d0 con 0x7f5eb80d2b50 2024-04-15T15:19:22.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eb5400640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb815c680 0x7f5eb815cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:22.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eb5e00640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb80d2b50 0x7f5eb81f4560 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c4d0 tx=0x7f5ea800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eae000640 1 -- 172.21.15.49:0/1591352914 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8018070 con 0x7f5eb80d2b50 2024-04-15T15:19:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eae000640 1 -- 172.21.15.49:0/1591352914 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea8007500 con 0x7f5eb80d2b50 2024-04-15T15:19:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eae000640 1 -- 172.21.15.49:0/1591352914 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8004a20 con 0x7f5eb80d2b50 2024-04-15T15:19:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.236+0000 7f5eb7200640 1 -- 172.21.15.49:0/1591352914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb80c99c0 con 0x7f5eb80d2b50 2024-04-15T15:19:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.237+0000 7f5eb7200640 1 -- 172.21.15.49:0/1591352914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb815d550 con 0x7f5eb80d2b50 2024-04-15T15:19:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.238+0000 7f5eb7200640 1 -- 172.21.15.49:0/1591352914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb80755c0 con 0x7f5eb80d2b50 2024-04-15T15:19:22.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.239+0000 7f5eae000640 1 -- 172.21.15.49:0/1591352914 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ea801b070 con 0x7f5eb80d2b50 2024-04-15T15:19:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.240+0000 7f5eae000640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5e9806c8f0 0x7f5e9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.240+0000 7f5eae000640 1 -- 172.21.15.49:0/1591352914 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5ea808ae50 con 0x7f5eb80d2b50 2024-04-15T15:19:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.240+0000 7f5eb5400640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5e9806c8f0 0x7f5e9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.241+0000 7f5eb5400640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5e9806c8f0 0x7f5e9806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4036000 tx=0x7f5ea4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.245+0000 7f5eae000640 1 -- 172.21.15.49:0/1591352914 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5eb81562d0 con 0x7f5eb80d2b50 2024-04-15T15:19:22.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.448+0000 7f5eb7200640 1 -- 172.21.15.49:0/1591352914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5eb80b9c40 con 0x7f5eb80d2b50 2024-04-15T15:19:22.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.449+0000 7f5eae000640 1 -- 172.21.15.49:0/1591352914 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f5ea8059280 con 0x7f5eb80d2b50 2024-04-15T15:19:22.450 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.451+0000 7f5eb7200640 1 -- 172.21.15.49:0/1591352914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5e9806c8f0 msgr2=0x7f5e9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.451+0000 7f5eb7200640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5e9806c8f0 0x7f5e9806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4036000 tx=0x7f5ea4008030 comp rx=0 tx=0).stop 2024-04-15T15:19:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.451+0000 7f5eb7200640 1 -- 172.21.15.49:0/1591352914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb80d2b50 msgr2=0x7f5eb81f4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.451+0000 7f5eb7200640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb80d2b50 0x7f5eb81f4560 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c4d0 tx=0x7f5ea800c990 comp rx=0 tx=0).stop 2024-04-15T15:19:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.452+0000 7f5eb7200640 1 -- 172.21.15.49:0/1591352914 shutdown_connections 2024-04-15T15:19:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.452+0000 7f5eb7200640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5e9806c8f0 0x7f5e9806edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.452+0000 7f5eb7200640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5eb815c680 0x7f5eb815cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.452+0000 7f5eb7200640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8162730 0x7f5eb8155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.452+0000 7f5eb7200640 1 --2- 172.21.15.49:0/1591352914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5eb80d2b50 0x7f5eb81f4560 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.452+0000 7f5eb7200640 1 -- 172.21.15.49:0/1591352914 >> 172.21.15.49:0/1591352914 conn(0x7f5eb8073480 msgr2=0x7f5eb80cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.452+0000 7f5eb7200640 1 -- 172.21.15.49:0/1591352914 shutdown_connections 2024-04-15T15:19:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.452+0000 7f5eb7200640 1 -- 172.21.15.49:0/1591352914 wait complete. 2024-04-15T15:19:22.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:22.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.699+0000 7f2edec00640 1 Processor -- start 2024-04-15T15:19:22.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.699+0000 7f2edec00640 1 -- start start 2024-04-15T15:19:22.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.700+0000 7f2edec00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8162020 0x7f2ed8162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.700+0000 7f2edec00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed80d0090 con 0x7f2ed80d1c90 2024-04-15T15:19:22.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.700+0000 7f2edec00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed80cfad0 con 0x7f2ed80cf110 2024-04-15T15:19:22.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.700+0000 7f2edec00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed8163440 con 0x7f2ed8162020 2024-04-15T15:19:22.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.701+0000 7f2ede200640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2ed80d1c90 0x7f2ed80d3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41810/0 (socket says 172.21.15.49:41810) 2024-04-15T15:19:22.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.701+0000 7f2ede200640 1 -- 172.21.15.49:0/312701209 learned_addr learned my addr 172.21.15.49:0/312701209 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.701+0000 7f2edd800640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8162020 0x7f2ed8162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.701+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2630533820 0 0) 0x7f2ed0001490 con 0x7f2ed80d1c90 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.701+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ec0003520 con 0x7f2ed80d1c90 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.701+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1592180483 0 0) 0x7f2ed80cfad0 con 0x7f2ed80cf110 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.701+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ed0001490 con 0x7f2ed80cf110 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4201574317 0 0) 0x7f2ec0003520 con 0x7f2ed80d1c90 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8162020 msgr2=0x7f2ed8162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8162020 0x7f2ed8162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 >> v1:172.21.15.107:6789/0 conn(0x7f2ed80cf110 legacy=0x7f2ed80d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed80cfdb0 con 0x7f2ed80d1c90 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f2ed0001bc0 con 0x7f2ed80d1c90 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 >> v1:172.21.15.49:6789/0 conn(0x7f2ed80d1c90 legacy=0x7f2ed80d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 0x7f2ec0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8162020 0x7f2ec0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec0004d40 0x7f2ec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8163440 con 0x7f2ec0003b70 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec000ba70 con 0x7f2ec0004d40 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec000bc70 con 0x7f2ed8162020 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.702+0000 7f2edd800640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 0x7f2ec0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.703+0000 7f2ede200640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec0004d40 0x7f2ec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.703+0000 7f2edce00640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8162020 0x7f2ec0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.703+0000 7f2edd800640 1 -- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8162020 msgr2=0x7f2ec0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.703+0000 7f2edd800640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8162020 0x7f2ec0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.703+0000 7f2edd800640 1 -- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec0004d40 msgr2=0x7f2ec00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:22.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.703+0000 7f2edd800640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec0004d40 0x7f2ec00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.703+0000 7f2edd800640 1 -- 172.21.15.49:0/312701209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec0003520 con 0x7f2ec0003b70 2024-04-15T15:19:22.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.704+0000 7f2edd800640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 0x7f2ec0003f70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009ac0 tx=0x7f2ec0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5701fcce2af21f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.704+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc039070 con 0x7f2ec0003b70 2024-04-15T15:19:22.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.704+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ecc02c6b0 con 0x7f2ec0003b70 2024-04-15T15:19:22.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.704+0000 7f2ed7e00640 1 -- 172.21.15.49:0/312701209 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc009690 con 0x7f2ec0003b70 2024-04-15T15:19:22.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.705+0000 7f2edec00640 1 -- 172.21.15.49:0/312701209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 msgr2=0x7f2ec0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.705+0000 7f2edec00640 1 --2- 172.21.15.49:0/312701209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 0x7f2ec0003f70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009ac0 tx=0x7f2ec0005830 comp rx=0 tx=0).stop 2024-04-15T15:19:22.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.705+0000 7f2ede200640 1 -- 172.21.15.49:0/312701209 reap_dead start 2024-04-15T15:19:22.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.705+0000 7f2edec00640 1 -- 172.21.15.49:0/312701209 shutdown_connections 2024-04-15T15:19:22.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.705+0000 7f2edec00640 1 -- 172.21.15.49:0/312701209 >> 172.21.15.49:0/312701209 conn(0x7f2ed806cf00 msgr2=0x7f2ed806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:22.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.705+0000 7f2edec00640 1 -- 172.21.15.49:0/312701209 shutdown_connections 2024-04-15T15:19:22.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.705+0000 7f2edec00640 1 -- 172.21.15.49:0/312701209 wait complete. 2024-04-15T15:19:22.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.707+0000 7f2edec00640 1 Processor -- start 2024-04-15T15:19:22.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.707+0000 7f2edec00640 1 -- start start 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.708+0000 7f2edec00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 0x7f2ed8145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.708+0000 7f2edec00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8152db0 0x7f2ed80c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.708+0000 7f2edec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed80c98e0 0x7f2ed80c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.708+0000 7f2edec00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed80cf880 con 0x7f2ec0003b70 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.708+0000 7f2edec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed80b7b30 con 0x7f2ed80c98e0 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.708+0000 7f2edec00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed80d2e70 con 0x7f2ed8152db0 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.708+0000 7f2edd800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 0x7f2ed8145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.708+0000 7f2edd800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 0x7f2ed8145b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34016/0 (socket says 172.21.15.49:34016) 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.708+0000 7f2edd800640 1 -- 172.21.15.49:0/705236337 learned_addr learned my addr 172.21.15.49:0/705236337 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.709+0000 7f2ede200640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed80c98e0 0x7f2ed80c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.709+0000 7f2edce00640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8152db0 0x7f2ed80c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.709+0000 7f2edd800640 1 -- 172.21.15.49:0/705236337 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8152db0 msgr2=0x7f2ed80c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.709+0000 7f2edd800640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8152db0 0x7f2ed80c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.709+0000 7f2edd800640 1 -- 172.21.15.49:0/705236337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed80c98e0 msgr2=0x7f2ed80c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.709+0000 7f2edd800640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed80c98e0 0x7f2ed80c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.709+0000 7f2edd800640 1 -- 172.21.15.49:0/705236337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed8146080 con 0x7f2ec0003b70 2024-04-15T15:19:22.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.709+0000 7f2ede200640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed80c98e0 0x7f2ed80c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:22.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.709+0000 7f2edce00640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8152db0 0x7f2ed80c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:22.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.709+0000 7f2edd800640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 0x7f2ed8145b40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009ac0 tx=0x7f2ecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.710+0000 7f2ed5600640 1 -- 172.21.15.49:0/705236337 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc039050 con 0x7f2ec0003b70 2024-04-15T15:19:22.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.710+0000 7f2ed5600640 1 -- 172.21.15.49:0/705236337 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ecc045070 con 0x7f2ec0003b70 2024-04-15T15:19:22.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.710+0000 7f2ed5600640 1 -- 172.21.15.49:0/705236337 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc003a50 con 0x7f2ec0003b70 2024-04-15T15:19:22.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.710+0000 7f2edec00640 1 -- 172.21.15.49:0/705236337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed8160880 con 0x7f2ec0003b70 2024-04-15T15:19:22.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.710+0000 7f2edec00640 1 -- 172.21.15.49:0/705236337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed81f4dd0 con 0x7f2ec0003b70 2024-04-15T15:19:22.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.711+0000 7f2edec00640 1 -- 172.21.15.49:0/705236337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed80755c0 con 0x7f2ec0003b70 2024-04-15T15:19:22.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.712+0000 7f2ed5600640 1 -- 172.21.15.49:0/705236337 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ecc003bf0 con 0x7f2ec0003b70 2024-04-15T15:19:22.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.713+0000 7f2ed5600640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2eb806c8f0 0x7f2eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:22.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.713+0000 7f2ed5600640 1 -- 172.21.15.49:0/705236337 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2ecc0ad7e0 con 0x7f2ec0003b70 2024-04-15T15:19:22.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.713+0000 7f2edce00640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2eb806c8f0 0x7f2eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:22.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.714+0000 7f2edce00640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2eb806c8f0 0x7f2eb806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2ed815e090 tx=0x7f2ec4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:22.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.717+0000 7f2ed5600640 1 -- 172.21.15.49:0/705236337 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ecc04e020 con 0x7f2ec0003b70 2024-04-15T15:19:22.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.924+0000 7f2edec00640 1 -- 172.21.15.49:0/705236337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ed80d25a0 con 0x7f2ec0003b70 2024-04-15T15:19:22.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.926+0000 7f2ed5600640 1 -- 172.21.15.49:0/705236337 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7f2ecc07bc30 con 0x7f2ec0003b70 2024-04-15T15:19:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:22.929 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:22.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.929+0000 7f2edec00640 1 -- 172.21.15.49:0/705236337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2eb806c8f0 msgr2=0x7f2eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.929+0000 7f2edec00640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2eb806c8f0 0x7f2eb806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2ed815e090 tx=0x7f2ec4009040 comp rx=0 tx=0).stop 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.929+0000 7f2edec00640 1 -- 172.21.15.49:0/705236337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 msgr2=0x7f2ed8145b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.929+0000 7f2edec00640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 0x7f2ed8145b40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009ac0 tx=0x7f2ecc004730 comp rx=0 tx=0).stop 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.929+0000 7f2edec00640 1 -- 172.21.15.49:0/705236337 shutdown_connections 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.929+0000 7f2edec00640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2eb806c8f0 0x7f2eb806edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.929+0000 7f2edec00640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ed8152db0 0x7f2ed80c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.929+0000 7f2edec00640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed80c98e0 0x7f2ed80c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.929+0000 7f2edec00640 1 --2- 172.21.15.49:0/705236337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0003b70 0x7f2ed8145b40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.929+0000 7f2edec00640 1 -- 172.21.15.49:0/705236337 >> 172.21.15.49:0/705236337 conn(0x7f2ed806cf00 msgr2=0x7f2ed80ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.930+0000 7f2edec00640 1 -- 172.21.15.49:0/705236337 shutdown_connections 2024-04-15T15:19:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:22.930+0000 7f2edec00640 1 -- 172.21.15.49:0/705236337 wait complete. 2024-04-15T15:19:22.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:22.950 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:23.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.190+0000 7efde0600640 1 Processor -- start 2024-04-15T15:19:23.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.190+0000 7efde0600640 1 -- start start 2024-04-15T15:19:23.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.191+0000 7efde0600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdd815a440 0x7efdd815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.191+0000 7efde0600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efdd8072150 0x7efdd80725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.191+0000 7efde0600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd815d520 con 0x7efdd806fc80 2024-04-15T15:19:23.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.192+0000 7efde0600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7efdd8072b00 con 0x7efdd8072150 2024-04-15T15:19:23.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.192+0000 7efde0600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7efdd806e280 con 0x7efdd815a440 2024-04-15T15:19:23.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.192+0000 7efdde600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efdd8072150 0x7efdd80725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.192+0000 7efdde600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efdd8072150 0x7efdd80725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34030/0 (socket says 172.21.15.49:34030) 2024-04-15T15:19:23.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.192+0000 7efdde600640 1 -- 172.21.15.49:0/1959164352 learned_addr learned my addr 172.21.15.49:0/1959164352 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:23.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.192+0000 7efdddc00640 1 --2- 172.21.15.49:0/1959164352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdd815a440 0x7efdd815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.193+0000 7efdde600640 1 -- 172.21.15.49:0/1959164352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdd815a440 msgr2=0x7efdd815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:23.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.193+0000 7efdde600640 1 --2- 172.21.15.49:0/1959164352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdd815a440 0x7efdd815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.193+0000 7efdde600640 1 -- 172.21.15.49:0/1959164352 >> v1:172.21.15.107:6789/0 conn(0x7efdd806fc80 legacy=0x7efdd80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:23.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.193+0000 7efdde600640 1 -- 172.21.15.49:0/1959164352 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd815d800 con 0x7efdd8072150 2024-04-15T15:19:23.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.194+0000 7efdde600640 1 --2- 172.21.15.49:0/1959164352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd8072150 0x7efdd80725c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009e50 tx=0x7efdd002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5b8addaa883a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.194+0000 7efdd7e00640 1 -- 172.21.15.49:0/1959164352 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd002cb00 con 0x7efdd8072150 2024-04-15T15:19:23.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.194+0000 7efdd7e00640 1 -- 172.21.15.49:0/1959164352 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd002cca0 con 0x7efdd8072150 2024-04-15T15:19:23.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.194+0000 7efdd7e00640 1 -- 172.21.15.49:0/1959164352 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0034610 con 0x7efdd8072150 2024-04-15T15:19:23.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.195+0000 7efde0600640 1 -- 172.21.15.49:0/1959164352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd8072150 msgr2=0x7efdd80725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.195+0000 7efde0600640 1 --2- 172.21.15.49:0/1959164352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd8072150 0x7efdd80725c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009e50 tx=0x7efdd002bf60 comp rx=0 tx=0).stop 2024-04-15T15:19:23.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.195+0000 7efde0600640 1 -- 172.21.15.49:0/1959164352 shutdown_connections 2024-04-15T15:19:23.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.195+0000 7efde0600640 1 --2- 172.21.15.49:0/1959164352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd8072150 0x7efdd80725c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.195+0000 7efde0600640 1 --2- 172.21.15.49:0/1959164352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdd815a440 0x7efdd815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.195+0000 7efde0600640 1 -- 172.21.15.49:0/1959164352 >> 172.21.15.49:0/1959164352 conn(0x7efdd80b2460 msgr2=0x7efdd80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:23.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.195+0000 7efde0600640 1 -- 172.21.15.49:0/1959164352 shutdown_connections 2024-04-15T15:19:23.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.195+0000 7efde0600640 1 -- 172.21.15.49:0/1959164352 wait complete. 2024-04-15T15:19:23.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.197+0000 7efde0600640 1 Processor -- start 2024-04-15T15:19:23.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.197+0000 7efde0600640 1 -- start start 2024-04-15T15:19:23.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.198+0000 7efde0600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdd806fc80 0x7efdd81326d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.198+0000 7efde0600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd8072150 0x7efdd80b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.198+0000 7efde0600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd815a440 0x7efdd815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.198+0000 7efde0600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efdd815a820 con 0x7efdd815a440 2024-04-15T15:19:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.198+0000 7efde0600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efdd80cd7f0 con 0x7efdd8072150 2024-04-15T15:19:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.198+0000 7efde0600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7efdd815ddc0 con 0x7efdd806fc80 2024-04-15T15:19:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.198+0000 7efdde600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd815a440 0x7efdd815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.198+0000 7efdde600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd815a440 0x7efdd815c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34036/0 (socket says 172.21.15.49:34036) 2024-04-15T15:19:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.198+0000 7efdde600640 1 -- 172.21.15.49:0/1964459734 learned_addr learned my addr 172.21.15.49:0/1964459734 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.199+0000 7efdddc00640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdd806fc80 0x7efdd81326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.199+0000 7efddd200640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd8072150 0x7efdd80b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.199+0000 7efdde600640 1 -- 172.21.15.49:0/1964459734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdd806fc80 msgr2=0x7efdd81326d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.199+0000 7efdde600640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdd806fc80 0x7efdd81326d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.199+0000 7efdde600640 1 -- 172.21.15.49:0/1964459734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd8072150 msgr2=0x7efdd80b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.199+0000 7efdde600640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd8072150 0x7efdd80b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.199+0000 7efdde600640 1 -- 172.21.15.49:0/1964459734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd81e5220 con 0x7efdd815a440 2024-04-15T15:19:23.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.199+0000 7efdde600640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd815a440 0x7efdd815c380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009e50 tx=0x7efdd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.200+0000 7efdd5600640 1 -- 172.21.15.49:0/1964459734 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0043020 con 0x7efdd815a440 2024-04-15T15:19:23.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.200+0000 7efdd5600640 1 -- 172.21.15.49:0/1964459734 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd002ce50 con 0x7efdd815a440 2024-04-15T15:19:23.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.200+0000 7efdd5600640 1 -- 172.21.15.49:0/1964459734 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0009510 con 0x7efdd815a440 2024-04-15T15:19:23.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.200+0000 7efde0600640 1 -- 172.21.15.49:0/1964459734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd81e5500 con 0x7efdd815a440 2024-04-15T15:19:23.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.200+0000 7efde0600640 1 -- 172.21.15.49:0/1964459734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd81e5ac0 con 0x7efdd815a440 2024-04-15T15:19:23.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.201+0000 7efde0600640 1 -- 172.21.15.49:0/1964459734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd8068c80 con 0x7efdd815a440 2024-04-15T15:19:23.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.202+0000 7efdd5600640 1 -- 172.21.15.49:0/1964459734 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdd0046020 con 0x7efdd815a440 2024-04-15T15:19:23.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.203+0000 7efdd5600640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efdbc06c8f0 0x7efdbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.203+0000 7efdd5600640 1 -- 172.21.15.49:0/1964459734 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efdd00ae210 con 0x7efdd815a440 2024-04-15T15:19:23.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.203+0000 7efdddc00640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efdbc06c8f0 0x7efdbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.204+0000 7efdddc00640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efdbc06c8f0 0x7efdbc06edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efdcc006fd0 tx=0x7efdcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.208+0000 7efdd5600640 1 -- 172.21.15.49:0/1964459734 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdd004d030 con 0x7efdd815a440 2024-04-15T15:19:23.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.411+0000 7efde0600640 1 -- 172.21.15.49:0/1964459734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efdd80b2840 con 0x7efdd815a440 2024-04-15T15:19:23.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.413+0000 7efdd5600640 1 -- 172.21.15.49:0/1964459734 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7efdd007c5a0 con 0x7efdd815a440 2024-04-15T15:19:23.414 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:23.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 -- 172.21.15.49:0/1964459734 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efdbc06c8f0 msgr2=0x7efdbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efdbc06c8f0 0x7efdbc06edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efdcc006fd0 tx=0x7efdcc008040 comp rx=0 tx=0).stop 2024-04-15T15:19:23.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 -- 172.21.15.49:0/1964459734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd815a440 msgr2=0x7efdd815c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd815a440 0x7efdd815c380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009e50 tx=0x7efdd0004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:23.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 -- 172.21.15.49:0/1964459734 shutdown_connections 2024-04-15T15:19:23.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efdbc06c8f0 0x7efdbc06edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdd815a440 0x7efdd815c380 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd8072150 0x7efdd80b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 --2- 172.21.15.49:0/1964459734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdd806fc80 0x7efdd81326d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 -- 172.21.15.49:0/1964459734 >> 172.21.15.49:0/1964459734 conn(0x7efdd80b2460 msgr2=0x7efdd80ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:23.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 -- 172.21.15.49:0/1964459734 shutdown_connections 2024-04-15T15:19:23.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.415+0000 7efde0600640 1 -- 172.21.15.49:0/1964459734 wait complete. 2024-04-15T15:19:23.436 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:23.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.666+0000 7f014b000640 1 Processor -- start 2024-04-15T15:19:23.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.667+0000 7f014b000640 1 -- start start 2024-04-15T15:19:23.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.668+0000 7f014b000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f014415c680 0x7f0144162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.668+0000 7f014b000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01440b70d0 con 0x7f01440d2b50 2024-04-15T15:19:23.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.668+0000 7f014b000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0144162dd0 con 0x7f01440b6c70 2024-04-15T15:19:23.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.668+0000 7f014b000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f01441630b0 con 0x7f014415c680 2024-04-15T15:19:23.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.669+0000 7f014a600640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f01440d2b50 0x7f01440d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41820/0 (socket says 172.21.15.49:41820) 2024-04-15T15:19:23.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.669+0000 7f014a600640 1 -- 172.21.15.49:0/1108908395 learned_addr learned my addr 172.21.15.49:0/1108908395 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:23.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.669+0000 7f0149c00640 1 --2- 172.21.15.49:0/1108908395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f014415c680 0x7f0144162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.669+0000 7f0143e00640 1 -- 172.21.15.49:0/1108908395 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 209219705 0 0) 0x7f01440b70d0 con 0x7f01440d2b50 2024-04-15T15:19:23.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.670+0000 7f0143e00640 1 -- 172.21.15.49:0/1108908395 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f012c003520 con 0x7f01440d2b50 2024-04-15T15:19:23.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.670+0000 7f0149c00640 1 -- 172.21.15.49:0/1108908395 >> v1:172.21.15.107:6789/0 conn(0x7f01440b6c70 legacy=0x7f014415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.670+0000 7f0149c00640 1 -- 172.21.15.49:0/1108908395 >> v1:172.21.15.49:6789/0 conn(0x7f01440d2b50 legacy=0x7f01440d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.670+0000 7f0143e00640 1 -- 172.21.15.49:0/1108908395 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3476781251 0 0) 0x7f012c003520 con 0x7f01440d2b50 2024-04-15T15:19:23.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.670+0000 7f0149c00640 1 -- 172.21.15.49:0/1108908395 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0144163390 con 0x7f014415c680 2024-04-15T15:19:23.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.671+0000 7f0149c00640 1 --2- 172.21.15.49:0/1108908395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f014415c680 0x7f0144162730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01380097d0 tx=0x7f013802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9a6b49ff8c4c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.672+0000 7f0143e00640 1 -- 172.21.15.49:0/1108908395 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138037070 con 0x7f014415c680 2024-04-15T15:19:23.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.672+0000 7f0143e00640 1 -- 172.21.15.49:0/1108908395 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0138030510 con 0x7f014415c680 2024-04-15T15:19:23.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.672+0000 7f0143e00640 1 -- 172.21.15.49:0/1108908395 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c0019f0 con 0x7f014415c680 2024-04-15T15:19:23.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.672+0000 7f014b000640 1 -- 172.21.15.49:0/1108908395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f014415c680 msgr2=0x7f0144162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.672+0000 7f014b000640 1 --2- 172.21.15.49:0/1108908395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f014415c680 0x7f0144162730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01380097d0 tx=0x7f013802b800 comp rx=0 tx=0).stop 2024-04-15T15:19:23.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.673+0000 7f014b000640 1 -- 172.21.15.49:0/1108908395 shutdown_connections 2024-04-15T15:19:23.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.673+0000 7f014b000640 1 --2- 172.21.15.49:0/1108908395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f014415c680 0x7f0144162730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.673+0000 7f014b000640 1 -- 172.21.15.49:0/1108908395 >> 172.21.15.49:0/1108908395 conn(0x7f0144073480 msgr2=0x7f0144073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:23.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.673+0000 7f014b000640 1 -- 172.21.15.49:0/1108908395 shutdown_connections 2024-04-15T15:19:23.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.673+0000 7f014b000640 1 -- 172.21.15.49:0/1108908395 wait complete. 2024-04-15T15:19:23.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.674+0000 7f014b000640 1 Processor -- start 2024-04-15T15:19:23.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.675+0000 7f014b000640 1 -- start start 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.675+0000 7f014b000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01440b6c70 0x7f014414abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.675+0000 7f014b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01440d2b50 0x7f01440d3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.675+0000 7f014b000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415c680 0x7f01440d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.675+0000 7f014b000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f014415ff60 con 0x7f014415c680 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.675+0000 7f014b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f014415ca60 con 0x7f01440d2b50 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.675+0000 7f014b000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f01440d1e80 con 0x7f01440b6c70 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.676+0000 7f014a600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415c680 0x7f01440d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.676+0000 7f014a600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415c680 0x7f01440d3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34046/0 (socket says 172.21.15.49:34046) 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.676+0000 7f014a600640 1 -- 172.21.15.49:0/1765094569 learned_addr learned my addr 172.21.15.49:0/1765094569 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.676+0000 7f0149c00640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01440b6c70 0x7f014414abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.676+0000 7f0149200640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01440d2b50 0x7f01440d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.676+0000 7f014a600640 1 -- 172.21.15.49:0/1765094569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01440b6c70 msgr2=0x7f014414abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.676+0000 7f014a600640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01440b6c70 0x7f014414abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.676+0000 7f014a600640 1 -- 172.21.15.49:0/1765094569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01440d2b50 msgr2=0x7f01440d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.676+0000 7f014a600640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01440d2b50 0x7f01440d3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.676+0000 7f014a600640 1 -- 172.21.15.49:0/1765094569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014414b690 con 0x7f014415c680 2024-04-15T15:19:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.677+0000 7f014a600640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415c680 0x7f01440d3c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f013c0018c0 tx=0x7f013c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.677+0000 7f0141600640 1 -- 172.21.15.49:0/1765094569 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c012400 con 0x7f014415c680 2024-04-15T15:19:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.677+0000 7f0141600640 1 -- 172.21.15.49:0/1765094569 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f013c0125c0 con 0x7f014415c680 2024-04-15T15:19:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.677+0000 7f014b000640 1 -- 172.21.15.49:0/1765094569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014414e2d0 con 0x7f014415c680 2024-04-15T15:19:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.677+0000 7f0141600640 1 -- 172.21.15.49:0/1765094569 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c0129b0 con 0x7f014415c680 2024-04-15T15:19:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.677+0000 7f014b000640 1 -- 172.21.15.49:0/1765094569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014414b120 con 0x7f014415c680 2024-04-15T15:19:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.678+0000 7f014b000640 1 -- 172.21.15.49:0/1765094569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01440755c0 con 0x7f014415c680 2024-04-15T15:19:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.679+0000 7f0141600640 1 -- 172.21.15.49:0/1765094569 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f013c012b50 con 0x7f014415c680 2024-04-15T15:19:23.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.680+0000 7f0141600640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f012406c8f0 0x7f012406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:23.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.680+0000 7f0141600640 1 -- 172.21.15.49:0/1765094569 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f013c08c340 con 0x7f014415c680 2024-04-15T15:19:23.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.681+0000 7f0149c00640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f012406c8f0 0x7f012406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:23.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.682+0000 7f0149c00640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f012406c8f0 0x7f012406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0138034000 tx=0x7f0138008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.685+0000 7f0141600640 1 -- 172.21.15.49:0/1765094569 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f013c0182b0 con 0x7f014415c680 2024-04-15T15:19:23.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.893+0000 7f014b000640 1 -- 172.21.15.49:0/1765094569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f014406dbe0 con 0x7f014415c680 2024-04-15T15:19:23.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.894+0000 7f0141600640 1 -- 172.21.15.49:0/1765094569 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f013c05a8c0 con 0x7f014415c680 2024-04-15T15:19:23.896 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 -- 172.21.15.49:0/1765094569 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f012406c8f0 msgr2=0x7f012406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f012406c8f0 0x7f012406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0138034000 tx=0x7f0138008030 comp rx=0 tx=0).stop 2024-04-15T15:19:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 -- 172.21.15.49:0/1765094569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415c680 msgr2=0x7f01440d3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415c680 0x7f01440d3c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f013c0018c0 tx=0x7f013c00a430 comp rx=0 tx=0).stop 2024-04-15T15:19:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 -- 172.21.15.49:0/1765094569 shutdown_connections 2024-04-15T15:19:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f012406c8f0 0x7f012406edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415c680 0x7f01440d3c70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01440d2b50 0x7f01440d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 --2- 172.21.15.49:0/1765094569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01440b6c70 0x7f014414abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 -- 172.21.15.49:0/1765094569 >> 172.21.15.49:0/1765094569 conn(0x7f0144073480 msgr2=0x7f0144073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 -- 172.21.15.49:0/1765094569 shutdown_connections 2024-04-15T15:19:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:23.897+0000 7f014b000640 1 -- 172.21.15.49:0/1765094569 wait complete. 2024-04-15T15:19:23.918 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.146+0000 7fa5c1800640 1 Processor -- start 2024-04-15T15:19:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.146+0000 7fa5c1800640 1 -- start start 2024-04-15T15:19:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.148+0000 7fa5c1800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa5bc0cebb0 0x7fa5bc167590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.148+0000 7fa5c1800640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5bc0cd2d0 con 0x7fa5bc0725f0 2024-04-15T15:19:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.148+0000 7fa5c1800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5bc0cd5b0 con 0x7fa5bc1461c0 2024-04-15T15:19:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.148+0000 7fa5c1800640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fa5bc0cf1d0 con 0x7fa5bc0cebb0 2024-04-15T15:19:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.148+0000 7fa5bb400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa5bc0725f0 0x7fa5bc0ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45838/0 (socket says 172.21.15.49:45838) 2024-04-15T15:19:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.148+0000 7fa5bb400640 1 -- 172.21.15.49:0/332540089 learned_addr learned my addr 172.21.15.49:0/332540089 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.149+0000 7fa5c0e00640 1 --2- 172.21.15.49:0/332540089 >> v2:172.21.15.107:3300/0 conn(0x7fa5bc0cebb0 0x7fa5bc167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.149+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 328985690 0 0) 0x7fa5bc0cd2d0 con 0x7fa5bc0725f0 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.149+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5a0003520 con 0x7fa5bc0725f0 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.149+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 327215285 0 0) 0x7fa5bc0cd5b0 con 0x7fa5bc1461c0 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.149+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5bc0cd2d0 con 0x7fa5bc1461c0 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.149+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2398389958 0 0) 0x7fa5a0003520 con 0x7fa5bc0725f0 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.149+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 >> v2:172.21.15.107:3300/0 conn(0x7fa5bc0cebb0 msgr2=0x7fa5bc167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.149+0000 7fa5baa00640 1 --2- 172.21.15.49:0/332540089 >> v2:172.21.15.107:3300/0 conn(0x7fa5bc0cebb0 0x7fa5bc167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.149+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 >> v1:172.21.15.107:6789/0 conn(0x7fa5bc1461c0 legacy=0x7fa5bc071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc0cd890 con 0x7fa5bc0725f0 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fa5a8001670 con 0x7fa5bc0725f0 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 >> v1:172.21.15.49:6789/0 conn(0x7fa5bc0725f0 legacy=0x7fa5bc0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5baa00640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5a0003b90 0x7fa5a0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5baa00640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5a0004800 0x7fa5a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5baa00640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a000c090 con 0x7fa5a00051c0 2024-04-15T15:19:24.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a000c210 con 0x7fa5a0003b90 2024-04-15T15:19:24.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a000c410 con 0x7fa5a0004800 2024-04-15T15:19:24.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5c0e00640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.150+0000 7fa5bbe00640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5a0003b90 0x7fa5a0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.151+0000 7fa5c0e00640 1 -- 172.21.15.49:0/332540089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5a0004800 msgr2=0x7fa5a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.151+0000 7fa5bb400640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5a0004800 0x7fa5a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.151+0000 7fa5c0e00640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5a0004800 0x7fa5a0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.151+0000 7fa5c0e00640 1 -- 172.21.15.49:0/332540089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5a0003b90 msgr2=0x7fa5a0003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.151+0000 7fa5c0e00640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5a0003b90 0x7fa5a0003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.151+0000 7fa5c0e00640 1 -- 172.21.15.49:0/332540089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a0003520 con 0x7fa5a00051c0 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.152+0000 7fa5c0e00640 1 -- 172.21.15.49:0/332540089 reap_dead start 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.152+0000 7fa5c0e00640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5a000b9c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009fc0 tx=0x7fa5a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2fa7d292a83e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.152+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4009c10 con 0x7fa5a00051c0 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.152+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b402cca0 con 0x7fa5a00051c0 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.152+0000 7fa5baa00640 1 -- 172.21.15.49:0/332540089 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4003bd0 con 0x7fa5a00051c0 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.153+0000 7fa5c1800640 1 -- 172.21.15.49:0/332540089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 msgr2=0x7fa5a000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.153+0000 7fa5c1800640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5a000b9c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009fc0 tx=0x7fa5a0005830 comp rx=0 tx=0).stop 2024-04-15T15:19:24.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.153+0000 7fa5c1800640 1 -- 172.21.15.49:0/332540089 shutdown_connections 2024-04-15T15:19:24.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.153+0000 7fa5c1800640 1 --2- 172.21.15.49:0/332540089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5a000b9c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.153+0000 7fa5c1800640 1 -- 172.21.15.49:0/332540089 >> 172.21.15.49:0/332540089 conn(0x7fa5bc06b930 msgr2=0x7fa5bc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:24.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.153+0000 7fa5c1800640 1 -- 172.21.15.49:0/332540089 shutdown_connections 2024-04-15T15:19:24.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.153+0000 7fa5c1800640 1 -- 172.21.15.49:0/332540089 wait complete. 2024-04-15T15:19:24.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.155+0000 7fa5c1800640 1 Processor -- start 2024-04-15T15:19:24.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.155+0000 7fa5c1800640 1 -- start start 2024-04-15T15:19:24.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.156+0000 7fa5c1800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5bc1562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.156+0000 7fa5c1800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5bc1f7a60 0x7fa5bc1f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.156+0000 7fa5c1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5bc15f3b0 0x7fa5bc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.156+0000 7fa5c1800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc0ce4a0 con 0x7fa5a00051c0 2024-04-15T15:19:24.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.156+0000 7fa5c1800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc0cd080 con 0x7fa5bc15f3b0 2024-04-15T15:19:24.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.156+0000 7fa5c1800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa5bc0ce620 con 0x7fa5bc1f7a60 2024-04-15T15:19:24.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.156+0000 7fa5bbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5bc1562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.156+0000 7fa5bbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5bc1562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47286/0 (socket says 172.21.15.49:47286) 2024-04-15T15:19:24.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.156+0000 7fa5bbe00640 1 -- 172.21.15.49:0/3345732489 learned_addr learned my addr 172.21.15.49:0/3345732489 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5c0e00640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5bc15f3b0 0x7fa5bc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5bb400640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5bc1f7a60 0x7fa5bc1f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5bbe00640 1 -- 172.21.15.49:0/3345732489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5bc1f7a60 msgr2=0x7fa5bc1f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5bbe00640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5bc1f7a60 0x7fa5bc1f7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5bbe00640 1 -- 172.21.15.49:0/3345732489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5bc15f3b0 msgr2=0x7fa5bc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5bbe00640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5bc15f3b0 0x7fa5bc15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5bbe00640 1 -- 172.21.15.49:0/3345732489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc1f8520 con 0x7fa5a00051c0 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5bbe00640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5bc1562e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00e500 tx=0x7fa5ac00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5b3e00640 1 -- 172.21.15.49:0/3345732489 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac0052e0 con 0x7fa5a00051c0 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5b3e00640 1 -- 172.21.15.49:0/3345732489 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5ac018070 con 0x7fa5a00051c0 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5b3e00640 1 -- 172.21.15.49:0/3345732489 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac003830 con 0x7fa5a00051c0 2024-04-15T15:19:24.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.157+0000 7fa5c1800640 1 -- 172.21.15.49:0/3345732489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5bc1f8800 con 0x7fa5a00051c0 2024-04-15T15:19:24.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.158+0000 7fa5c1800640 1 -- 172.21.15.49:0/3345732489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5bc13a4f0 con 0x7fa5a00051c0 2024-04-15T15:19:24.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.159+0000 7fa5c1800640 1 -- 172.21.15.49:0/3345732489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5bc13e0f0 con 0x7fa5a00051c0 2024-04-15T15:19:24.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.160+0000 7fa5b3e00640 1 -- 172.21.15.49:0/3345732489 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5ac0039d0 con 0x7fa5a00051c0 2024-04-15T15:19:24.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.161+0000 7fa5b3e00640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa59c06c8f0 0x7fa59c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.161+0000 7fa5b3e00640 1 -- 172.21.15.49:0/3345732489 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa5ac08b9f0 con 0x7fa5a00051c0 2024-04-15T15:19:24.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.161+0000 7fa5bb400640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa59c06c8f0 0x7fa59c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.162+0000 7fa5bb400640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa59c06c8f0 0x7fa59c06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8000c00 tx=0x7fa5a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:24.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.165+0000 7fa5b3e00640 1 -- 172.21.15.49:0/3345732489 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5ac059d00 con 0x7fa5a00051c0 2024-04-15T15:19:24.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.367+0000 7fa5c1800640 1 -- 172.21.15.49:0/3345732489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa5bc0755c0 con 0x7fa5a00051c0 2024-04-15T15:19:24.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.368+0000 7fa5b3e00640 1 -- 172.21.15.49:0/3345732489 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fa5ac059600 con 0x7fa5a00051c0 2024-04-15T15:19:24.370 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:24.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 -- 172.21.15.49:0/3345732489 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa59c06c8f0 msgr2=0x7fa59c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa59c06c8f0 0x7fa59c06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8000c00 tx=0x7fa5a8009040 comp rx=0 tx=0).stop 2024-04-15T15:19:24.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 -- 172.21.15.49:0/3345732489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 msgr2=0x7fa5bc1562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5bc1562e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00e500 tx=0x7fa5ac00e9c0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 -- 172.21.15.49:0/3345732489 shutdown_connections 2024-04-15T15:19:24.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa59c06c8f0 0x7fa59c06edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5bc1f7a60 0x7fa5bc1f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5bc15f3b0 0x7fa5bc15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 --2- 172.21.15.49:0/3345732489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a00051c0 0x7fa5bc1562e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 -- 172.21.15.49:0/3345732489 >> 172.21.15.49:0/3345732489 conn(0x7fa5bc06b930 msgr2=0x7fa5bc0b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:24.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 -- 172.21.15.49:0/3345732489 shutdown_connections 2024-04-15T15:19:24.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.371+0000 7fa5c1800640 1 -- 172.21.15.49:0/3345732489 wait complete. 2024-04-15T15:19:24.391 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:24.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.616+0000 7fb71b200640 1 Processor -- start 2024-04-15T15:19:24.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.617+0000 7fb71b200640 1 -- start start 2024-04-15T15:19:24.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.618+0000 7fb71b200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb71c162020 0x7fb71c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.618+0000 7fb71b200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb71c0d1c90 0x7fb71c0d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.618+0000 7fb71b200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71c0d0090 con 0x7fb71c0cf110 2024-04-15T15:19:24.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.618+0000 7fb71b200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb71c163440 con 0x7fb71c162020 2024-04-15T15:19:24.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.618+0000 7fb71b200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fb71c0b7b30 con 0x7fb71c0d1c90 2024-04-15T15:19:24.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.619+0000 7fb719e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb71c162020 0x7fb71c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.619+0000 7fb719e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb71c162020 0x7fb71c162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47296/0 (socket says 172.21.15.49:47296) 2024-04-15T15:19:24.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.619+0000 7fb719e00640 1 -- 172.21.15.49:0/125244330 learned_addr learned my addr 172.21.15.49:0/125244330 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:24.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.619+0000 7fb71a800640 1 --2- 172.21.15.49:0/125244330 >> v2:172.21.15.107:3300/0 conn(0x7fb71c0d1c90 0x7fb71c0d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.619+0000 7fb719e00640 1 -- 172.21.15.49:0/125244330 >> v2:172.21.15.107:3300/0 conn(0x7fb71c0d1c90 msgr2=0x7fb71c0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:24.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.619+0000 7fb719e00640 1 --2- 172.21.15.49:0/125244330 >> v2:172.21.15.107:3300/0 conn(0x7fb71c0d1c90 0x7fb71c0d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.619+0000 7fb719e00640 1 -- 172.21.15.49:0/125244330 >> v1:172.21.15.107:6789/0 conn(0x7fb71c0cf110 legacy=0x7fb71c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:24.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.619+0000 7fb719e00640 1 -- 172.21.15.49:0/125244330 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c0cfad0 con 0x7fb71c162020 2024-04-15T15:19:24.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.620+0000 7fb719e00640 1 --2- 172.21.15.49:0/125244330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 0x7fb71c162f00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb71c08d460 tx=0x7fb70c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd1c4c7ec570217 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:24.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.621+0000 7fb718a00640 1 -- 172.21.15.49:0/125244330 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c035020 con 0x7fb71c162020 2024-04-15T15:19:24.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.621+0000 7fb718a00640 1 -- 172.21.15.49:0/125244330 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70c031870 con 0x7fb71c162020 2024-04-15T15:19:24.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.621+0000 7fb718a00640 1 -- 172.21.15.49:0/125244330 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c031c40 con 0x7fb71c162020 2024-04-15T15:19:24.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.621+0000 7fb71b200640 1 -- 172.21.15.49:0/125244330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 msgr2=0x7fb71c162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.621+0000 7fb71b200640 1 --2- 172.21.15.49:0/125244330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 0x7fb71c162f00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb71c08d460 tx=0x7fb70c02b910 comp rx=0 tx=0).stop 2024-04-15T15:19:24.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.621+0000 7fb71b200640 1 -- 172.21.15.49:0/125244330 shutdown_connections 2024-04-15T15:19:24.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.621+0000 7fb71b200640 1 --2- 172.21.15.49:0/125244330 >> v2:172.21.15.107:3300/0 conn(0x7fb71c0d1c90 0x7fb71c0d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.621+0000 7fb71b200640 1 --2- 172.21.15.49:0/125244330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 0x7fb71c162f00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.621+0000 7fb71b200640 1 -- 172.21.15.49:0/125244330 >> 172.21.15.49:0/125244330 conn(0x7fb71c06cf00 msgr2=0x7fb71c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:24.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.622+0000 7fb71b200640 1 -- 172.21.15.49:0/125244330 shutdown_connections 2024-04-15T15:19:24.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.622+0000 7fb71b200640 1 -- 172.21.15.49:0/125244330 wait complete. 2024-04-15T15:19:24.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.623+0000 7fb71b200640 1 Processor -- start 2024-04-15T15:19:24.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.623+0000 7fb71b200640 1 -- start start 2024-04-15T15:19:24.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.624+0000 7fb71b200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71c0cf110 0x7fb71c0b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.624+0000 7fb71b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71c0d1c90 0x7fb71c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.624+0000 7fb71b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 0x7fb71c0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.624+0000 7fb71b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c0d2b90 con 0x7fb71c162020 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.624+0000 7fb71b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c0cf880 con 0x7fb71c0d1c90 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.624+0000 7fb71b200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c0d2e70 con 0x7fb71c0cf110 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb71a800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 0x7fb71c0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb71a800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 0x7fb71c0ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47310/0 (socket says 172.21.15.49:47310) 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb71a800640 1 -- 172.21.15.49:0/31114692 learned_addr learned my addr 172.21.15.49:0/31114692 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb719e00640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71c0cf110 0x7fb71c0b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb719400640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71c0d1c90 0x7fb71c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb71a800640 1 -- 172.21.15.49:0/31114692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71c0cf110 msgr2=0x7fb71c0b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb71a800640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71c0cf110 0x7fb71c0b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb71a800640 1 -- 172.21.15.49:0/31114692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71c0d1c90 msgr2=0x7fb71c0c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb71a800640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71c0d1c90 0x7fb71c0c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb71a800640 1 -- 172.21.15.49:0/31114692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c0c8d90 con 0x7fb71c162020 2024-04-15T15:19:24.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.625+0000 7fb71a800640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 0x7fb71c0ca280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb714007e30 tx=0x7fb71400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:24.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.626+0000 7fb712000640 1 -- 172.21.15.49:0/31114692 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714014620 con 0x7fb71c162020 2024-04-15T15:19:24.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.626+0000 7fb712000640 1 -- 172.21.15.49:0/31114692 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb714019070 con 0x7fb71c162020 2024-04-15T15:19:24.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.626+0000 7fb71b200640 1 -- 172.21.15.49:0/31114692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c0c9320 con 0x7fb71c162020 2024-04-15T15:19:24.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.626+0000 7fb712000640 1 -- 172.21.15.49:0/31114692 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714014a00 con 0x7fb71c162020 2024-04-15T15:19:24.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.626+0000 7fb71b200640 1 -- 172.21.15.49:0/31114692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c0c98e0 con 0x7fb71c162020 2024-04-15T15:19:24.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.627+0000 7fb71b200640 1 -- 172.21.15.49:0/31114692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c0755c0 con 0x7fb71c162020 2024-04-15T15:19:24.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.628+0000 7fb712000640 1 -- 172.21.15.49:0/31114692 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb714014ba0 con 0x7fb71c162020 2024-04-15T15:19:24.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.629+0000 7fb712000640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6fc06c8f0 0x7fb6fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:24.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.629+0000 7fb712000640 1 -- 172.21.15.49:0/31114692 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb71408c400 con 0x7fb71c162020 2024-04-15T15:19:24.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.629+0000 7fb719e00640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6fc06c8f0 0x7fb6fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:24.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.630+0000 7fb719e00640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6fc06c8f0 0x7fb6fc06edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb70c036000 tx=0x7fb70c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:24.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.633+0000 7fb712000640 1 -- 172.21.15.49:0/31114692 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb71405a820 con 0x7fb71c162020 2024-04-15T15:19:24.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.843+0000 7fb71b200640 1 -- 172.21.15.49:0/31114692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb71c0b7650 con 0x7fb71c162020 2024-04-15T15:19:24.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.846+0000 7fb712000640 1 -- 172.21.15.49:0/31114692 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7fb71405a120 con 0x7fb71c162020 2024-04-15T15:19:24.848 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:24.848 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:24.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.848+0000 7fb71b200640 1 -- 172.21.15.49:0/31114692 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6fc06c8f0 msgr2=0x7fb6fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.848+0000 7fb71b200640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6fc06c8f0 0x7fb6fc06edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb70c036000 tx=0x7fb70c03c040 comp rx=0 tx=0).stop 2024-04-15T15:19:24.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.848+0000 7fb71b200640 1 -- 172.21.15.49:0/31114692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 msgr2=0x7fb71c0ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:24.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.848+0000 7fb71b200640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 0x7fb71c0ca280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb714007e30 tx=0x7fb71400e4b0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.849+0000 7fb71b200640 1 -- 172.21.15.49:0/31114692 shutdown_connections 2024-04-15T15:19:24.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.849+0000 7fb71b200640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6fc06c8f0 0x7fb6fc06edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.849+0000 7fb71b200640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71c162020 0x7fb71c0ca280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.849+0000 7fb71b200640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71c0d1c90 0x7fb71c0c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.849+0000 7fb71b200640 1 --2- 172.21.15.49:0/31114692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71c0cf110 0x7fb71c0b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:24.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.849+0000 7fb71b200640 1 -- 172.21.15.49:0/31114692 >> 172.21.15.49:0/31114692 conn(0x7fb71c06cf00 msgr2=0x7fb71c0cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:24.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.849+0000 7fb71b200640 1 -- 172.21.15.49:0/31114692 shutdown_connections 2024-04-15T15:19:24.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:24.849+0000 7fb71b200640 1 -- 172.21.15.49:0/31114692 wait complete. 2024-04-15T15:19:24.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:24.873 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:25.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.105+0000 7f23dca00640 1 Processor -- start 2024-04-15T15:19:25.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.105+0000 7f23dca00640 1 -- start start 2024-04-15T15:19:25.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.106+0000 7f23dca00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f23d80b54b0 0x7f23d80b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.106+0000 7f23dca00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d80b4410 con 0x7f23d806f260 2024-04-15T15:19:25.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.106+0000 7f23dca00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d80b46f0 con 0x7f23d8145170 2024-04-15T15:19:25.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.106+0000 7f23dca00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f23d8145550 con 0x7f23d80b54b0 2024-04-15T15:19:25.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.107+0000 7f23d6800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f23d806f260 0x7f23d806f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45844/0 (socket says 172.21.15.49:45844) 2024-04-15T15:19:25.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.107+0000 7f23d6800640 1 -- 172.21.15.49:0/3132558823 learned_addr learned my addr 172.21.15.49:0/3132558823 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.107+0000 7f23cd400640 1 --2- 172.21.15.49:0/3132558823 >> v2:172.21.15.107:3300/0 conn(0x7f23d80b54b0 0x7f23d80b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.107+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1024783466 0 0) 0x7f23d80b4410 con 0x7f23d806f260 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.107+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23bc003520 con 0x7f23d806f260 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3132536211 0 0) 0x7f23bc003520 con 0x7f23d806f260 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 >> v2:172.21.15.107:3300/0 conn(0x7f23d80b54b0 msgr2=0x7f23d80b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 --2- 172.21.15.49:0/3132558823 >> v2:172.21.15.107:3300/0 conn(0x7f23d80b54b0 0x7f23d80b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 >> v1:172.21.15.107:6789/0 conn(0x7f23d8145170 legacy=0x7f23d80cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d80b49d0 con 0x7f23d806f260 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f23d00019f0 con 0x7f23d806f260 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 >> v1:172.21.15.49:6789/0 conn(0x7f23d806f260 legacy=0x7f23d806f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23bc0039b0 0x7f23bc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23bc004800 0x7f23bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23bc0051c0 0x7f23bc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f23d8145550 con 0x7f23bc0039b0 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc00e2a0 con 0x7f23bc0051c0 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f23bc00e4a0 con 0x7f23bc004800 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.108+0000 7f23d5e00640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23bc0039b0 0x7f23bc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.109+0000 7f23d6800640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23bc0051c0 0x7f23bc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.109+0000 7f23d5e00640 1 -- 172.21.15.49:0/3132558823 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23bc004800 msgr2=0x7f23bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.109+0000 7f23d5e00640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23bc004800 0x7f23bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.109+0000 7f23d5e00640 1 -- 172.21.15.49:0/3132558823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23bc0051c0 msgr2=0x7f23bc00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.109+0000 7f23d5e00640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23bc0051c0 0x7f23bc00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.109+0000 7f23d5e00640 1 -- 172.21.15.49:0/3132558823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc003520 con 0x7f23bc0039b0 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.109+0000 7f23d6800640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23bc0051c0 0x7f23bc00dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:25.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.109+0000 7f23d6800640 1 -- 172.21.15.49:0/3132558823 reap_dead start 2024-04-15T15:19:25.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.110+0000 7f23d5e00640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23bc0039b0 0x7f23bc003df0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f23c8001750 tx=0x7f23bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfefb08b42a3d09b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:25.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.110+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8001370 con 0x7f23bc0039b0 2024-04-15T15:19:25.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.110+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c80052d0 con 0x7f23bc0039b0 2024-04-15T15:19:25.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.110+0000 7f23d5400640 1 -- 172.21.15.49:0/3132558823 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8003830 con 0x7f23bc0039b0 2024-04-15T15:19:25.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.111+0000 7f23dca00640 1 -- 172.21.15.49:0/3132558823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23bc0039b0 msgr2=0x7f23bc003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.111+0000 7f23dca00640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23bc0039b0 0x7f23bc003df0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f23c8001750 tx=0x7f23bc005830 comp rx=0 tx=0).stop 2024-04-15T15:19:25.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.111+0000 7f23dca00640 1 -- 172.21.15.49:0/3132558823 shutdown_connections 2024-04-15T15:19:25.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.111+0000 7f23dca00640 1 --2- 172.21.15.49:0/3132558823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23bc0039b0 0x7f23bc003df0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.111+0000 7f23dca00640 1 -- 172.21.15.49:0/3132558823 >> 172.21.15.49:0/3132558823 conn(0x7f23d80b2460 msgr2=0x7f23d80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:25.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.111+0000 7f23dca00640 1 -- 172.21.15.49:0/3132558823 shutdown_connections 2024-04-15T15:19:25.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.111+0000 7f23dca00640 1 -- 172.21.15.49:0/3132558823 wait complete. 2024-04-15T15:19:25.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.113+0000 7f23dca00640 1 Processor -- start 2024-04-15T15:19:25.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.113+0000 7f23dca00640 1 -- start start 2024-04-15T15:19:25.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.114+0000 7f23dca00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23bc0039b0 0x7f23d81e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.114+0000 7f23dca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23d81e0f80 0x7f23d806b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.114+0000 7f23dca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23d81441e0 0x7f23d806be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.114+0000 7f23dca00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f23d80b4030 con 0x7f23d81e0f80 2024-04-15T15:19:25.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.114+0000 7f23dca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23d80b4f90 con 0x7f23d81441e0 2024-04-15T15:19:25.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.114+0000 7f23dca00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f23d80b41b0 con 0x7f23bc0039b0 2024-04-15T15:19:25.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.114+0000 7f23cd400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23d81e0f80 0x7f23d806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.114+0000 7f23cd400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23d81e0f80 0x7f23d806b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47326/0 (socket says 172.21.15.49:47326) 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.114+0000 7f23cd400640 1 -- 172.21.15.49:0/3035960409 learned_addr learned my addr 172.21.15.49:0/3035960409 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23d5e00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23bc0039b0 0x7f23d81e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23d6800640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23d81441e0 0x7f23d806be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23cd400640 1 -- 172.21.15.49:0/3035960409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23bc0039b0 msgr2=0x7f23d81e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23cd400640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23bc0039b0 0x7f23d81e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23cd400640 1 -- 172.21.15.49:0/3035960409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23d81441e0 msgr2=0x7f23d806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23cd400640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23d81441e0 0x7f23d806be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23cd400640 1 -- 172.21.15.49:0/3035960409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d81e1410 con 0x7f23d81e0f80 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23d5e00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23bc0039b0 0x7f23d81e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23d6800640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23d81441e0 0x7f23d806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:25.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23cd400640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23d81e0f80 0x7f23d806b930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f23c000e890 tx=0x7f23c000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:25.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23cea00640 1 -- 172.21.15.49:0/3035960409 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0017070 con 0x7f23d81e0f80 2024-04-15T15:19:25.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23cea00640 1 -- 172.21.15.49:0/3035960409 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c000c940 con 0x7f23d81e0f80 2024-04-15T15:19:25.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23cea00640 1 -- 172.21.15.49:0/3035960409 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c000cd10 con 0x7f23d81e0f80 2024-04-15T15:19:25.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.115+0000 7f23dca00640 1 -- 172.21.15.49:0/3035960409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d81e16f0 con 0x7f23d81e0f80 2024-04-15T15:19:25.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.116+0000 7f23dca00640 1 -- 172.21.15.49:0/3035960409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d81e1cb0 con 0x7f23d81e0f80 2024-04-15T15:19:25.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.117+0000 7f23dca00640 1 -- 172.21.15.49:0/3035960409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d8068c80 con 0x7f23d81e0f80 2024-04-15T15:19:25.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.118+0000 7f23cea00640 1 -- 172.21.15.49:0/3035960409 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23c001a070 con 0x7f23d81e0f80 2024-04-15T15:19:25.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.119+0000 7f23cea00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f23b406c920 0x7f23b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.119+0000 7f23cea00640 1 -- 172.21.15.49:0/3035960409 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f23c008bda0 con 0x7f23d81e0f80 2024-04-15T15:19:25.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.119+0000 7f23d5e00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f23b406c920 0x7f23b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.120+0000 7f23d5e00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f23b406c920 0x7f23b406ede0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f23c8001750 tx=0x7f23c8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:25.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.123+0000 7f23cea00640 1 -- 172.21.15.49:0/3035960409 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23c005a130 con 0x7f23d81e0f80 2024-04-15T15:19:25.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.328+0000 7f23dca00640 1 -- 172.21.15.49:0/3035960409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23d815ba60 con 0x7f23d81e0f80 2024-04-15T15:19:25.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.329+0000 7f23cea00640 1 -- 172.21.15.49:0/3035960409 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f23c0059a30 con 0x7f23d81e0f80 2024-04-15T15:19:25.331 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:25.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 -- 172.21.15.49:0/3035960409 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f23b406c920 msgr2=0x7f23b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f23b406c920 0x7f23b406ede0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f23c8001750 tx=0x7f23c8009030 comp rx=0 tx=0).stop 2024-04-15T15:19:25.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 -- 172.21.15.49:0/3035960409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23d81e0f80 msgr2=0x7f23d806b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23d81e0f80 0x7f23d806b930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f23c000e890 tx=0x7f23c000ed50 comp rx=0 tx=0).stop 2024-04-15T15:19:25.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 -- 172.21.15.49:0/3035960409 shutdown_connections 2024-04-15T15:19:25.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f23b406c920 0x7f23b406ede0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23d81441e0 0x7f23d806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23d81e0f80 0x7f23d806b930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 --2- 172.21.15.49:0/3035960409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23bc0039b0 0x7f23d81e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 -- 172.21.15.49:0/3035960409 >> 172.21.15.49:0/3035960409 conn(0x7f23d80b2460 msgr2=0x7f23d80ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:25.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 -- 172.21.15.49:0/3035960409 shutdown_connections 2024-04-15T15:19:25.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.332+0000 7f23dca00640 1 -- 172.21.15.49:0/3035960409 wait complete. 2024-04-15T15:19:25.353 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:25.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.584+0000 7f48aaa00640 1 Processor -- start 2024-04-15T15:19:25.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.584+0000 7f48aaa00640 1 -- start start 2024-04-15T15:19:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.586+0000 7f48aaa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f48a4163520 0x7f48a415f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.586+0000 7f48aaa00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48a415ffe0 con 0x7f48a4145070 2024-04-15T15:19:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.587+0000 7f48aaa00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48a4162a40 con 0x7f48a4072860 2024-04-15T15:19:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.587+0000 7f48aaa00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f48a40afde0 con 0x7f48a4163520 2024-04-15T15:19:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.587+0000 7f48a3e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f48a4145070 0x7f48a4072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45846/0 (socket says 172.21.15.49:45846) 2024-04-15T15:19:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.587+0000 7f48a3e00640 1 -- 172.21.15.49:0/4285789948 learned_addr learned my addr 172.21.15.49:0/4285789948 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.587+0000 7f48a8a00640 1 --2- 172.21.15.49:0/4285789948 >> v2:172.21.15.49:3300/0 conn(0x7f48a4163520 0x7f48a415f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.587+0000 7f48a2a00640 1 -- 172.21.15.49:0/4285789948 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2585598109 0 0) 0x7f48a415ffe0 con 0x7f48a4145070 2024-04-15T15:19:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.587+0000 7f48a2a00640 1 -- 172.21.15.49:0/4285789948 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4888003520 con 0x7f48a4145070 2024-04-15T15:19:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.587+0000 7f48a8a00640 1 -- 172.21.15.49:0/4285789948 >> v1:172.21.15.107:6789/0 conn(0x7f48a4072860 legacy=0x7f48a415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.587+0000 7f48a8a00640 1 -- 172.21.15.49:0/4285789948 >> v1:172.21.15.49:6789/0 conn(0x7f48a4145070 legacy=0x7f48a4072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.587+0000 7f48a8a00640 1 -- 172.21.15.49:0/4285789948 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a4162d20 con 0x7f48a4163520 2024-04-15T15:19:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.589+0000 7f48a8a00640 1 --2- 172.21.15.49:0/4285789948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4163520 0x7f48a415f8b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f489c009dc0 tx=0x7f489c02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5c79c2a50a6a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.589+0000 7f48a2a00640 1 -- 172.21.15.49:0/4285789948 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c039070 con 0x7f48a4163520 2024-04-15T15:19:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.589+0000 7f48a2a00640 1 -- 172.21.15.49:0/4285789948 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489c02cc40 con 0x7f48a4163520 2024-04-15T15:19:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.589+0000 7f48a2a00640 1 -- 172.21.15.49:0/4285789948 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c034440 con 0x7f48a4163520 2024-04-15T15:19:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.589+0000 7f48aaa00640 1 -- 172.21.15.49:0/4285789948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4163520 msgr2=0x7f48a415f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.589+0000 7f48aaa00640 1 --2- 172.21.15.49:0/4285789948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4163520 0x7f48a415f8b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f489c009dc0 tx=0x7f489c02be90 comp rx=0 tx=0).stop 2024-04-15T15:19:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.590+0000 7f48aaa00640 1 -- 172.21.15.49:0/4285789948 shutdown_connections 2024-04-15T15:19:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.590+0000 7f48aaa00640 1 --2- 172.21.15.49:0/4285789948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4163520 0x7f48a415f8b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.590+0000 7f48aaa00640 1 -- 172.21.15.49:0/4285789948 >> 172.21.15.49:0/4285789948 conn(0x7f48a40b2460 msgr2=0x7f48a40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.590+0000 7f48aaa00640 1 -- 172.21.15.49:0/4285789948 shutdown_connections 2024-04-15T15:19:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.590+0000 7f48aaa00640 1 -- 172.21.15.49:0/4285789948 wait complete. 2024-04-15T15:19:25.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.591+0000 7f48aaa00640 1 Processor -- start 2024-04-15T15:19:25.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.592+0000 7f48aaa00640 1 -- start start 2024-04-15T15:19:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.592+0000 7f48aaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4072860 0x7f48a41410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.592+0000 7f48aaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a4145070 0x7f48a415def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48aaa00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48a4163520 0x7f48a41613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48aaa00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4145450 con 0x7f48a4072860 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48aaa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48a41627f0 con 0x7f48a4145070 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48aaa00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f48a4169b80 con 0x7f48a4163520 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4072860 0x7f48a41410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4072860 0x7f48a41410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47340/0 (socket says 172.21.15.49:47340) 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a3e00640 1 -- 172.21.15.49:0/444778572 learned_addr learned my addr 172.21.15.49:0/444778572 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a3400640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a4145070 0x7f48a415def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a8a00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48a4163520 0x7f48a41613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a3e00640 1 -- 172.21.15.49:0/444778572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48a4163520 msgr2=0x7f48a41613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a3e00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48a4163520 0x7f48a41613a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a3e00640 1 -- 172.21.15.49:0/444778572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a4145070 msgr2=0x7f48a415def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a3e00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a4145070 0x7f48a415def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a3e00640 1 -- 172.21.15.49:0/444778572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a4148860 con 0x7f48a4072860 2024-04-15T15:19:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.593+0000 7f48a3400640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a4145070 0x7f48a415def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.594+0000 7f48a3e00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4072860 0x7f48a41410e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f489400c500 tx=0x7f489400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.594+0000 7f489be00640 1 -- 172.21.15.49:0/444778572 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894017070 con 0x7f48a4072860 2024-04-15T15:19:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.594+0000 7f489be00640 1 -- 172.21.15.49:0/444778572 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4894005ce0 con 0x7f48a4072860 2024-04-15T15:19:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.594+0000 7f489be00640 1 -- 172.21.15.49:0/444778572 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894007330 con 0x7f48a4072860 2024-04-15T15:19:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.594+0000 7f48aaa00640 1 -- 172.21.15.49:0/444778572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a415e430 con 0x7f48a4072860 2024-04-15T15:19:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.594+0000 7f48aaa00640 1 -- 172.21.15.49:0/444778572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a41482a0 con 0x7f48a4072860 2024-04-15T15:19:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.595+0000 7f48aaa00640 1 -- 172.21.15.49:0/444778572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a4068c80 con 0x7f48a4072860 2024-04-15T15:19:25.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.596+0000 7f489be00640 1 -- 172.21.15.49:0/444778572 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f489401a070 con 0x7f48a4072860 2024-04-15T15:19:25.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.597+0000 7f489be00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488406c8f0 0x7f488406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:25.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.597+0000 7f489be00640 1 -- 172.21.15.49:0/444778572 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f489408afe0 con 0x7f48a4072860 2024-04-15T15:19:25.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.597+0000 7f48a3400640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488406c8f0 0x7f488406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:25.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.598+0000 7f48a3400640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488406c8f0 0x7f488406edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f48a406e3a0 tx=0x7f4890009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:25.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.602+0000 7f489be00640 1 -- 172.21.15.49:0/444778572 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48940594e0 con 0x7f48a4072860 2024-04-15T15:19:25.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.806+0000 7f48aaa00640 1 -- 172.21.15.49:0/444778572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48a40b28e0 con 0x7f48a4072860 2024-04-15T15:19:25.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.807+0000 7f489be00640 1 -- 172.21.15.49:0/444778572 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f4894058de0 con 0x7f48a4072860 2024-04-15T15:19:25.809 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 -- 172.21.15.49:0/444778572 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488406c8f0 msgr2=0x7f488406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488406c8f0 0x7f488406edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f48a406e3a0 tx=0x7f4890009040 comp rx=0 tx=0).stop 2024-04-15T15:19:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 -- 172.21.15.49:0/444778572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4072860 msgr2=0x7f48a41410e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4072860 0x7f48a41410e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f489400c500 tx=0x7f489400c9c0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 -- 172.21.15.49:0/444778572 shutdown_connections 2024-04-15T15:19:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488406c8f0 0x7f488406edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48a4163520 0x7f48a41613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a4145070 0x7f48a415def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 --2- 172.21.15.49:0/444778572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a4072860 0x7f48a41410e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:25.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 -- 172.21.15.49:0/444778572 >> 172.21.15.49:0/444778572 conn(0x7f48a40b2460 msgr2=0x7f48a406ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:25.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 -- 172.21.15.49:0/444778572 shutdown_connections 2024-04-15T15:19:25.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:25.810+0000 7f48aaa00640 1 -- 172.21.15.49:0/444778572 wait complete. 2024-04-15T15:19:25.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:26.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.060+0000 7f2823200640 1 Processor -- start 2024-04-15T15:19:26.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.060+0000 7f2823200640 1 -- start start 2024-04-15T15:19:26.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.062+0000 7f2823200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f281c1461c0 0x7f281c071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.062+0000 7f2823200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f281c0cd2d0 con 0x7f281c072420 2024-04-15T15:19:26.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.062+0000 7f2823200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f281c0cd5b0 con 0x7f281c0cebb0 2024-04-15T15:19:26.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.062+0000 7f2823200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f281c0cf1d0 con 0x7f281c1461c0 2024-04-15T15:19:26.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.062+0000 7f2821400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f281c072420 0x7f281c0ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45854/0 (socket says 172.21.15.49:45854) 2024-04-15T15:19:26.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.062+0000 7f2821e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f281c1461c0 0x7f281c071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.062+0000 7f2821400640 1 -- 172.21.15.49:0/2619551752 learned_addr learned my addr 172.21.15.49:0/2619551752 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:26.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.063+0000 7f2821e00640 1 -- 172.21.15.49:0/2619551752 >> v1:172.21.15.107:6789/0 conn(0x7f281c0cebb0 legacy=0x7f281c06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:26.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.063+0000 7f2821e00640 1 -- 172.21.15.49:0/2619551752 >> v1:172.21.15.49:6789/0 conn(0x7f281c072420 legacy=0x7f281c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.063+0000 7f2820a00640 1 -- 172.21.15.49:0/2619551752 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 419774822 0 0) 0x7f281c0cd2d0 con 0x7f281c072420 2024-04-15T15:19:26.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.063+0000 7f2821e00640 1 -- 172.21.15.49:0/2619551752 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c0cd890 con 0x7f281c1461c0 2024-04-15T15:19:26.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.064+0000 7f2821e00640 1 --2- 172.21.15.49:0/2619551752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1461c0 0x7f281c071ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28100097a0 tx=0x7f281002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f139fc8b67022c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.064+0000 7f2820a00640 1 -- 172.21.15.49:0/2619551752 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810035020 con 0x7f281c1461c0 2024-04-15T15:19:26.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.064+0000 7f2820a00640 1 -- 172.21.15.49:0/2619551752 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2810031830 con 0x7f281c1461c0 2024-04-15T15:19:26.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.064+0000 7f2820a00640 1 -- 172.21.15.49:0/2619551752 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810031c00 con 0x7f281c1461c0 2024-04-15T15:19:26.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.065+0000 7f2823200640 1 -- 172.21.15.49:0/2619551752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1461c0 msgr2=0x7f281c071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.065+0000 7f2823200640 1 --2- 172.21.15.49:0/2619551752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1461c0 0x7f281c071ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28100097a0 tx=0x7f281002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.065+0000 7f2823200640 1 -- 172.21.15.49:0/2619551752 shutdown_connections 2024-04-15T15:19:26.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.065+0000 7f2823200640 1 --2- 172.21.15.49:0/2619551752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1461c0 0x7f281c071ee0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.065+0000 7f2823200640 1 -- 172.21.15.49:0/2619551752 >> 172.21.15.49:0/2619551752 conn(0x7f281c06b930 msgr2=0x7f281c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:26.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.065+0000 7f2823200640 1 -- 172.21.15.49:0/2619551752 shutdown_connections 2024-04-15T15:19:26.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.065+0000 7f2823200640 1 -- 172.21.15.49:0/2619551752 wait complete. 2024-04-15T15:19:26.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.067+0000 7f2823200640 1 Processor -- start 2024-04-15T15:19:26.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.067+0000 7f2823200640 1 -- start start 2024-04-15T15:19:26.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.068+0000 7f2823200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c072420 0x7f281c13ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.068+0000 7f2823200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c0cebb0 0x7f281c12e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.068+0000 7f2823200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f281c1461c0 0x7f281c156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.068+0000 7f2823200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f281c0cd080 con 0x7f281c0cebb0 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.068+0000 7f2823200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f281c06e630 con 0x7f281c072420 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.068+0000 7f2823200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f281c06e7b0 con 0x7f281c1461c0 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.068+0000 7f2821400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c0cebb0 0x7f281c12e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.068+0000 7f2821400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c0cebb0 0x7f281c12e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47348/0 (socket says 172.21.15.49:47348) 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.068+0000 7f2821400640 1 -- 172.21.15.49:0/385987149 learned_addr learned my addr 172.21.15.49:0/385987149 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2821e00640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c072420 0x7f281c13ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2821400640 1 -- 172.21.15.49:0/385987149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f281c1461c0 msgr2=0x7f281c156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2821400640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f281c1461c0 0x7f281c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2821400640 1 -- 172.21.15.49:0/385987149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c072420 msgr2=0x7f281c13ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2821400640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c072420 0x7f281c13ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2821400640 1 -- 172.21.15.49:0/385987149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c156650 con 0x7f281c0cebb0 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2821e00640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c072420 0x7f281c13ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:26.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2821400640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c0cebb0 0x7f281c12e5d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f280c00c4d0 tx=0x7f280c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2816000640 1 -- 172.21.15.49:0/385987149 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c018070 con 0x7f281c0cebb0 2024-04-15T15:19:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2816000640 1 -- 172.21.15.49:0/385987149 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280c007500 con 0x7f281c0cebb0 2024-04-15T15:19:26.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2816000640 1 -- 172.21.15.49:0/385987149 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c004a20 con 0x7f281c0cebb0 2024-04-15T15:19:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.069+0000 7f2823200640 1 -- 172.21.15.49:0/385987149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c143f40 con 0x7f281c0cebb0 2024-04-15T15:19:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.070+0000 7f2823200640 1 -- 172.21.15.49:0/385987149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c144500 con 0x7f281c0cebb0 2024-04-15T15:19:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.071+0000 7f2823200640 1 -- 172.21.15.49:0/385987149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281c0755c0 con 0x7f281c0cebb0 2024-04-15T15:19:26.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.072+0000 7f2816000640 1 -- 172.21.15.49:0/385987149 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f280c01b070 con 0x7f281c0cebb0 2024-04-15T15:19:26.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.073+0000 7f2816000640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f280006c8f0 0x7f280006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.073+0000 7f2816000640 1 -- 172.21.15.49:0/385987149 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f280c08af80 con 0x7f281c0cebb0 2024-04-15T15:19:26.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.073+0000 7f2821e00640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f280006c8f0 0x7f280006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.074+0000 7f2821e00640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f280006c8f0 0x7f280006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2810036000 tx=0x7f2810008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.078+0000 7f2816000640 1 -- 172.21.15.49:0/385987149 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f280c059280 con 0x7f281c0cebb0 2024-04-15T15:19:26.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.281+0000 7f2823200640 1 -- 172.21.15.49:0/385987149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f281c072e40 con 0x7f281c0cebb0 2024-04-15T15:19:26.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.282+0000 7f2816000640 1 -- 172.21.15.49:0/385987149 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f280c058b80 con 0x7f281c0cebb0 2024-04-15T15:19:26.284 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:26.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 -- 172.21.15.49:0/385987149 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f280006c8f0 msgr2=0x7f280006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f280006c8f0 0x7f280006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2810036000 tx=0x7f2810008030 comp rx=0 tx=0).stop 2024-04-15T15:19:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 -- 172.21.15.49:0/385987149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c0cebb0 msgr2=0x7f281c12e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c0cebb0 0x7f281c12e5d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f280c00c4d0 tx=0x7f280c00c990 comp rx=0 tx=0).stop 2024-04-15T15:19:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 -- 172.21.15.49:0/385987149 shutdown_connections 2024-04-15T15:19:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f280006c8f0 0x7f280006edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f281c1461c0 0x7f281c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c0cebb0 0x7f281c12e5d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 --2- 172.21.15.49:0/385987149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c072420 0x7f281c13ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 -- 172.21.15.49:0/385987149 >> 172.21.15.49:0/385987149 conn(0x7f281c06b930 msgr2=0x7f281c15d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 -- 172.21.15.49:0/385987149 shutdown_connections 2024-04-15T15:19:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.285+0000 7f2823200640 1 -- 172.21.15.49:0/385987149 wait complete. 2024-04-15T15:19:26.308 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.536+0000 7fe98a600640 1 Processor -- start 2024-04-15T15:19:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.537+0000 7fe98a600640 1 -- start start 2024-04-15T15:19:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.538+0000 7fe98a600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9841552e0 0x7fe9840afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.538+0000 7fe98a600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe98415c170 0x7fe984072a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.538+0000 7fe98a600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9840b48e0 con 0x7fe9841556c0 2024-04-15T15:19:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.538+0000 7fe98a600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe9840b38d0 con 0x7fe9841552e0 2024-04-15T15:19:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.538+0000 7fe98a600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fe9840b3b20 con 0x7fe98415c170 2024-04-15T15:19:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.539+0000 7fe983400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9841552e0 0x7fe9840afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.539+0000 7fe983400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9841552e0 0x7fe9840afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47360/0 (socket says 172.21.15.49:47360) 2024-04-15T15:19:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.539+0000 7fe983400640 1 -- 172.21.15.49:0/1753553462 learned_addr learned my addr 172.21.15.49:0/1753553462 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.539+0000 7fe982a00640 1 --2- 172.21.15.49:0/1753553462 >> v2:172.21.15.107:3300/0 conn(0x7fe98415c170 0x7fe984072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.539+0000 7fe983400640 1 -- 172.21.15.49:0/1753553462 >> v2:172.21.15.107:3300/0 conn(0x7fe98415c170 msgr2=0x7fe984072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.539+0000 7fe983400640 1 --2- 172.21.15.49:0/1753553462 >> v2:172.21.15.107:3300/0 conn(0x7fe98415c170 0x7fe984072a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.539+0000 7fe983400640 1 -- 172.21.15.49:0/1753553462 >> v1:172.21.15.107:6789/0 conn(0x7fe9841556c0 legacy=0x7fe98415ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.539+0000 7fe983400640 1 -- 172.21.15.49:0/1753553462 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9840b3e00 con 0x7fe9841552e0 2024-04-15T15:19:26.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.540+0000 7fe983400640 1 --2- 172.21.15.49:0/1753553462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9841552e0 0x7fe9840afb60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9840b56a0 tx=0x7fe97402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5980424380fd146a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.541+0000 7fe982000640 1 -- 172.21.15.49:0/1753553462 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974035020 con 0x7fe9841552e0 2024-04-15T15:19:26.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.541+0000 7fe982000640 1 -- 172.21.15.49:0/1753553462 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe974031870 con 0x7fe9841552e0 2024-04-15T15:19:26.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.541+0000 7fe982000640 1 -- 172.21.15.49:0/1753553462 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974031c40 con 0x7fe9841552e0 2024-04-15T15:19:26.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.542+0000 7fe98a600640 1 -- 172.21.15.49:0/1753553462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9841552e0 msgr2=0x7fe9840afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.542+0000 7fe98a600640 1 --2- 172.21.15.49:0/1753553462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9841552e0 0x7fe9840afb60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9840b56a0 tx=0x7fe97402b910 comp rx=0 tx=0).stop 2024-04-15T15:19:26.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.542+0000 7fe98a600640 1 -- 172.21.15.49:0/1753553462 shutdown_connections 2024-04-15T15:19:26.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.542+0000 7fe98a600640 1 --2- 172.21.15.49:0/1753553462 >> v2:172.21.15.107:3300/0 conn(0x7fe98415c170 0x7fe984072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.542+0000 7fe98a600640 1 --2- 172.21.15.49:0/1753553462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9841552e0 0x7fe9840afb60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.542+0000 7fe98a600640 1 -- 172.21.15.49:0/1753553462 >> 172.21.15.49:0/1753553462 conn(0x7fe9840b27b0 msgr2=0x7fe9840b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:26.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.542+0000 7fe98a600640 1 -- 172.21.15.49:0/1753553462 shutdown_connections 2024-04-15T15:19:26.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.542+0000 7fe98a600640 1 -- 172.21.15.49:0/1753553462 wait complete. 2024-04-15T15:19:26.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.544+0000 7fe98a600640 1 Processor -- start 2024-04-15T15:19:26.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.544+0000 7fe98a600640 1 -- start start 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe98a600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9841552e0 0x7fe9841f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe98a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9841556c0 0x7fe9841f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe98a600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe98415c170 0x7fe9840af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe98a600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe9840d0010 con 0x7fe98415c170 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe98a600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9840b43c0 con 0x7fe9841556c0 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe98a600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe9840d0190 con 0x7fe9841552e0 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe983e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe98415c170 0x7fe9840af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe983e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe98415c170 0x7fe9840af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47374/0 (socket says 172.21.15.49:47374) 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe983e00640 1 -- 172.21.15.49:0/594021650 learned_addr learned my addr 172.21.15.49:0/594021650 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe983400640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9841552e0 0x7fe9841f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.545+0000 7fe982a00640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9841556c0 0x7fe9841f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe983e00640 1 -- 172.21.15.49:0/594021650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9841552e0 msgr2=0x7fe9841f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe983e00640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9841552e0 0x7fe9841f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe983e00640 1 -- 172.21.15.49:0/594021650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9841556c0 msgr2=0x7fe9841f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe983e00640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9841556c0 0x7fe9841f10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe983e00640 1 -- 172.21.15.49:0/594021650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9841f42c0 con 0x7fe98415c170 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe983e00640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe98415c170 0x7fe9840af0d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00c6d0 tx=0x7fe97c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe97b400640 1 -- 172.21.15.49:0/594021650 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c016020 con 0x7fe98415c170 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe97b400640 1 -- 172.21.15.49:0/594021650 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97c00a840 con 0x7fe98415c170 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe97b400640 1 -- 172.21.15.49:0/594021650 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00ac70 con 0x7fe98415c170 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe98a600640 1 -- 172.21.15.49:0/594021650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9841f45a0 con 0x7fe98415c170 2024-04-15T15:19:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.546+0000 7fe98a600640 1 -- 172.21.15.49:0/594021650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe984158d20 con 0x7fe98415c170 2024-04-15T15:19:26.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.548+0000 7fe98a600640 1 -- 172.21.15.49:0/594021650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9841f4ac0 con 0x7fe98415c170 2024-04-15T15:19:26.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.548+0000 7fe97b400640 1 -- 172.21.15.49:0/594021650 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe97c019020 con 0x7fe98415c170 2024-04-15T15:19:26.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.549+0000 7fe97b400640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe96406c8f0 0x7fe96406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:26.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.549+0000 7fe97b400640 1 -- 172.21.15.49:0/594021650 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe97c08c040 con 0x7fe98415c170 2024-04-15T15:19:26.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.550+0000 7fe983400640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe96406c8f0 0x7fe96406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:26.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.550+0000 7fe983400640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe96406c8f0 0x7fe96406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe974036000 tx=0x7fe97403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:26.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.554+0000 7fe97b400640 1 -- 172.21.15.49:0/594021650 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9841f4ac0 con 0x7fe98415c170 2024-04-15T15:19:26.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.760+0000 7fe98a600640 1 -- 172.21.15.49:0/594021650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe984068c80 con 0x7fe98415c170 2024-04-15T15:19:26.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.762+0000 7fe97b400640 1 -- 172.21.15.49:0/594021650 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7fe97c020020 con 0x7fe98415c170 2024-04-15T15:19:26.764 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:26.765 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:26.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 -- 172.21.15.49:0/594021650 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe96406c8f0 msgr2=0x7fe96406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe96406c8f0 0x7fe96406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe974036000 tx=0x7fe97403c040 comp rx=0 tx=0).stop 2024-04-15T15:19:26.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 -- 172.21.15.49:0/594021650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe98415c170 msgr2=0x7fe9840af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:26.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe98415c170 0x7fe9840af0d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00c6d0 tx=0x7fe97c00cb90 comp rx=0 tx=0).stop 2024-04-15T15:19:26.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 -- 172.21.15.49:0/594021650 shutdown_connections 2024-04-15T15:19:26.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe96406c8f0 0x7fe96406edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe98415c170 0x7fe9840af0d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9841556c0 0x7fe9841f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 --2- 172.21.15.49:0/594021650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9841552e0 0x7fe9841f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:26.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 -- 172.21.15.49:0/594021650 >> 172.21.15.49:0/594021650 conn(0x7fe9840b27b0 msgr2=0x7fe98415c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:26.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 -- 172.21.15.49:0/594021650 shutdown_connections 2024-04-15T15:19:26.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:26.765+0000 7fe98a600640 1 -- 172.21.15.49:0/594021650 wait complete. 2024-04-15T15:19:26.789 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:26.789 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:27.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.020+0000 7f65d2c00640 1 Processor -- start 2024-04-15T15:19:27.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.021+0000 7f65d2c00640 1 -- start start 2024-04-15T15:19:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.022+0000 7f65d2c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f65cc0d26b0 0x7f65cc0b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.022+0000 7f65d2c00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65cc0cfef0 con 0x7f65cc161f80 2024-04-15T15:19:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.022+0000 7f65d2c00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65cc0d2bb0 con 0x7f65cc0b7e80 2024-04-15T15:19:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.022+0000 7f65d2c00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f65cc0d2e90 con 0x7f65cc0d26b0 2024-04-15T15:19:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.023+0000 7f65d1800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f65cc161f80 0x7f65cc0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45866/0 (socket says 172.21.15.49:45866) 2024-04-15T15:19:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.023+0000 7f65d1800640 1 -- 172.21.15.49:0/221568312 learned_addr learned my addr 172.21.15.49:0/221568312 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.023+0000 7f65d0e00640 1 --2- 172.21.15.49:0/221568312 >> v2:172.21.15.107:3300/0 conn(0x7f65cc0d26b0 0x7f65cc0b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.023+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2618090564 0 0) 0x7f65cc0cfef0 con 0x7f65cc161f80 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.023+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65b4003520 con 0x7f65cc161f80 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.023+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 431059533 0 0) 0x7f65b4003520 con 0x7f65cc161f80 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.023+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 >> v2:172.21.15.107:3300/0 conn(0x7f65cc0d26b0 msgr2=0x7f65cc0b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.023+0000 7f65cbe00640 1 --2- 172.21.15.49:0/221568312 >> v2:172.21.15.107:3300/0 conn(0x7f65cc0d26b0 0x7f65cc0b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 >> v1:172.21.15.107:6789/0 conn(0x7f65cc0b7e80 legacy=0x7f65cc0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc0d3170 con 0x7f65cc161f80 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f65c0001670 con 0x7f65cc161f80 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 >> v1:172.21.15.49:6789/0 conn(0x7f65cc161f80 legacy=0x7f65cc0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65cbe00640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4003a00 0x7f65b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65cbe00640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b4004800 0x7f65b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65cbe00640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65b40051c0 0x7f65b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc0d2e90 con 0x7f65b4004800 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f65b400c090 con 0x7f65b4003a00 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f65b400c290 con 0x7f65b40051c0 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65d0e00640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b4004800 0x7f65b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65d1800640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4003a00 0x7f65b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.024+0000 7f65d2200640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65b40051c0 0x7f65b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.025+0000 7f65d0e00640 1 -- 172.21.15.49:0/221568312 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65b40051c0 msgr2=0x7f65b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.025+0000 7f65d0e00640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65b40051c0 0x7f65b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.025+0000 7f65d0e00640 1 -- 172.21.15.49:0/221568312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4003a00 msgr2=0x7f65b4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.025+0000 7f65d0e00640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4003a00 0x7f65b4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.025+0000 7f65d0e00640 1 -- 172.21.15.49:0/221568312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b4003520 con 0x7f65b4004800 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.025+0000 7f65d2200640 1 -- 172.21.15.49:0/221568312 reap_dead start 2024-04-15T15:19:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.025+0000 7f65d1800640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4003a00 0x7f65b4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:27.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.026+0000 7f65d0e00640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b4004800 0x7f65b4004c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009a90 tx=0x7f65b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e56756d831ccac server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.026+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8039070 con 0x7f65b4004800 2024-04-15T15:19:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.026+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b802c680 con 0x7f65b4004800 2024-04-15T15:19:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.026+0000 7f65cbe00640 1 -- 172.21.15.49:0/221568312 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8009690 con 0x7f65b4004800 2024-04-15T15:19:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.027+0000 7f65d2c00640 1 -- 172.21.15.49:0/221568312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b4004800 msgr2=0x7f65b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.027+0000 7f65d2c00640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b4004800 0x7f65b4004c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009a90 tx=0x7f65b4005830 comp rx=0 tx=0).stop 2024-04-15T15:19:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.027+0000 7f65d2c00640 1 -- 172.21.15.49:0/221568312 shutdown_connections 2024-04-15T15:19:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.027+0000 7f65d2c00640 1 --2- 172.21.15.49:0/221568312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b4004800 0x7f65b4004c80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.027+0000 7f65d2c00640 1 -- 172.21.15.49:0/221568312 >> 172.21.15.49:0/221568312 conn(0x7f65cc06ce40 msgr2=0x7f65cc06d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.027+0000 7f65d2c00640 1 -- 172.21.15.49:0/221568312 shutdown_connections 2024-04-15T15:19:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.027+0000 7f65d2c00640 1 -- 172.21.15.49:0/221568312 wait complete. 2024-04-15T15:19:27.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.029+0000 7f65d2c00640 1 Processor -- start 2024-04-15T15:19:27.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.029+0000 7f65d2c00640 1 -- start start 2024-04-15T15:19:27.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.030+0000 7f65d2c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4004800 0x7f65cc0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.030+0000 7f65d2c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65cc15e200 0x7f65cc0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.030+0000 7f65d2c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65cc0caf90 0x7f65cc15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.030+0000 7f65d2c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc0b7940 con 0x7f65cc0caf90 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.030+0000 7f65d2c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc162360 con 0x7f65b4004800 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.030+0000 7f65d2c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f65cc0b7ac0 con 0x7f65cc15e200 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.030+0000 7f65d2200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65cc0caf90 0x7f65cc15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.030+0000 7f65d2200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65cc0caf90 0x7f65cc15be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47390/0 (socket says 172.21.15.49:47390) 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.030+0000 7f65d2200640 1 -- 172.21.15.49:0/2573748708 learned_addr learned my addr 172.21.15.49:0/2573748708 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65d1800640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4004800 0x7f65cc0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65d0e00640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65cc15e200 0x7f65cc0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65d2200640 1 -- 172.21.15.49:0/2573748708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65cc15e200 msgr2=0x7f65cc0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65d2200640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65cc15e200 0x7f65cc0caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65d2200640 1 -- 172.21.15.49:0/2573748708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4004800 msgr2=0x7f65cc0c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65d2200640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4004800 0x7f65cc0c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65d2200640 1 -- 172.21.15.49:0/2573748708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc0c7820 con 0x7f65cc0caf90 2024-04-15T15:19:27.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65d0e00640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65cc15e200 0x7f65cc0caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:27.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65d1800640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4004800 0x7f65cc0c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:27.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65d2200640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65cc0caf90 0x7f65cc15be40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0d4750 tx=0x7f65c400e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65c9600640 1 -- 172.21.15.49:0/2573748708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4003df0 con 0x7f65cc0caf90 2024-04-15T15:19:27.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65c9600640 1 -- 172.21.15.49:0/2573748708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c4013420 con 0x7f65cc0caf90 2024-04-15T15:19:27.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.031+0000 7f65c9600640 1 -- 172.21.15.49:0/2573748708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4013810 con 0x7f65cc0caf90 2024-04-15T15:19:27.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.032+0000 7f65d2c00640 1 -- 172.21.15.49:0/2573748708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc155430 con 0x7f65cc0caf90 2024-04-15T15:19:27.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.032+0000 7f65d2c00640 1 -- 172.21.15.49:0/2573748708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc1559f0 con 0x7f65cc0caf90 2024-04-15T15:19:27.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.033+0000 7f65d2c00640 1 -- 172.21.15.49:0/2573748708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65cc0755c0 con 0x7f65cc0caf90 2024-04-15T15:19:27.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.034+0000 7f65c9600640 1 -- 172.21.15.49:0/2573748708 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65c40139b0 con 0x7f65cc0caf90 2024-04-15T15:19:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.035+0000 7f65c9600640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f65ac06c920 0x7f65ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.035+0000 7f65c9600640 1 -- 172.21.15.49:0/2573748708 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f65c408bd10 con 0x7f65cc0caf90 2024-04-15T15:19:27.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.035+0000 7f65d1800640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f65ac06c920 0x7f65ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.036+0000 7f65d1800640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f65ac06c920 0x7f65ac06ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f65cc06a920 tx=0x7f65c0001520 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.040+0000 7f65c9600640 1 -- 172.21.15.49:0/2573748708 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65c4091020 con 0x7f65cc0caf90 2024-04-15T15:19:27.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.244+0000 7f65d2c00640 1 -- 172.21.15.49:0/2573748708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65cc068f20 con 0x7f65cc0caf90 2024-04-15T15:19:27.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.245+0000 7f65c9600640 1 -- 172.21.15.49:0/2573748708 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f65c405a130 con 0x7f65cc0caf90 2024-04-15T15:19:27.247 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 -- 172.21.15.49:0/2573748708 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f65ac06c920 msgr2=0x7f65ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f65ac06c920 0x7f65ac06ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f65cc06a920 tx=0x7f65c0001520 comp rx=0 tx=0).stop 2024-04-15T15:19:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 -- 172.21.15.49:0/2573748708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65cc0caf90 msgr2=0x7f65cc15be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65cc0caf90 0x7f65cc15be40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0d4750 tx=0x7f65c400e9e0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 -- 172.21.15.49:0/2573748708 shutdown_connections 2024-04-15T15:19:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f65ac06c920 0x7f65ac06ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65cc15e200 0x7f65cc0caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65cc0caf90 0x7f65cc15be40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 --2- 172.21.15.49:0/2573748708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b4004800 0x7f65cc0c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 -- 172.21.15.49:0/2573748708 >> 172.21.15.49:0/2573748708 conn(0x7f65cc06ce40 msgr2=0x7f65cc153700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 -- 172.21.15.49:0/2573748708 shutdown_connections 2024-04-15T15:19:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.248+0000 7f65d2c00640 1 -- 172.21.15.49:0/2573748708 wait complete. 2024-04-15T15:19:27.271 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.504+0000 7f297cc00640 1 Processor -- start 2024-04-15T15:19:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.504+0000 7f297cc00640 1 -- start start 2024-04-15T15:19:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.505+0000 7f297cc00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f297815c680 0x7f2978162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.506+0000 7f297cc00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f29780d2b50 0x7f29780d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.506+0000 7f297cc00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29780b70d0 con 0x7f29780b6c70 2024-04-15T15:19:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.506+0000 7f297cc00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f2978162dd0 con 0x7f297815c680 2024-04-15T15:19:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.506+0000 7f297cc00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f29781630b0 con 0x7f29780d2b50 2024-04-15T15:19:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.506+0000 7f2977400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f297815c680 0x7f2978162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.506+0000 7f2976a00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f29780b6c70 0x7f297815f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45882/0 (socket says 172.21.15.49:45882) 2024-04-15T15:19:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.506+0000 7f2977400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f297815c680 0x7f2978162730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47392/0 (socket says 172.21.15.49:47392) 2024-04-15T15:19:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.506+0000 7f2976a00640 1 -- 172.21.15.49:0/1802717740 learned_addr learned my addr 172.21.15.49:0/1802717740 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.506+0000 7f2977e00640 1 --2- 172.21.15.49:0/1802717740 >> v2:172.21.15.107:3300/0 conn(0x7f29780d2b50 0x7f29780d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.507+0000 7f2977400640 1 -- 172.21.15.49:0/1802717740 >> v2:172.21.15.107:3300/0 conn(0x7f29780d2b50 msgr2=0x7f29780d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.507+0000 7f2977400640 1 --2- 172.21.15.49:0/1802717740 >> v2:172.21.15.107:3300/0 conn(0x7f29780d2b50 0x7f29780d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.507+0000 7f2977400640 1 -- 172.21.15.49:0/1802717740 >> v1:172.21.15.49:6789/0 conn(0x7f29780b6c70 legacy=0x7f297815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.507+0000 7f2977400640 1 -- 172.21.15.49:0/1802717740 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2978163390 con 0x7f297815c680 2024-04-15T15:19:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.507+0000 7f2977e00640 1 --2- 172.21.15.49:0/1802717740 >> v2:172.21.15.107:3300/0 conn(0x7f29780d2b50 0x7f29780d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.508+0000 7f2977400640 1 --2- 172.21.15.49:0/1802717740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297815c680 0x7f2978162730 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f29780d1680 tx=0x7f296802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d338c20019b17d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.508+0000 7f2976000640 1 -- 172.21.15.49:0/1802717740 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968035020 con 0x7f297815c680 2024-04-15T15:19:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.508+0000 7f2976000640 1 -- 172.21.15.49:0/1802717740 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2968031830 con 0x7f297815c680 2024-04-15T15:19:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.508+0000 7f2976000640 1 -- 172.21.15.49:0/1802717740 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968031c00 con 0x7f297815c680 2024-04-15T15:19:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.509+0000 7f297cc00640 1 -- 172.21.15.49:0/1802717740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297815c680 msgr2=0x7f2978162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.509+0000 7f297cc00640 1 --2- 172.21.15.49:0/1802717740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297815c680 0x7f2978162730 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f29780d1680 tx=0x7f296802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.509+0000 7f297cc00640 1 -- 172.21.15.49:0/1802717740 shutdown_connections 2024-04-15T15:19:27.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.509+0000 7f297cc00640 1 --2- 172.21.15.49:0/1802717740 >> v2:172.21.15.107:3300/0 conn(0x7f29780d2b50 0x7f29780d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.509+0000 7f297cc00640 1 --2- 172.21.15.49:0/1802717740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f297815c680 0x7f2978162730 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.509+0000 7f297cc00640 1 -- 172.21.15.49:0/1802717740 >> 172.21.15.49:0/1802717740 conn(0x7f2978073480 msgr2=0x7f2978073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:27.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.509+0000 7f297cc00640 1 -- 172.21.15.49:0/1802717740 shutdown_connections 2024-04-15T15:19:27.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.509+0000 7f297cc00640 1 -- 172.21.15.49:0/1802717740 wait complete. 2024-04-15T15:19:27.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.511+0000 7f297cc00640 1 Processor -- start 2024-04-15T15:19:27.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.511+0000 7f297cc00640 1 -- start start 2024-04-15T15:19:27.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f297cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29780b6c70 0x7f29780cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f297cc00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29780d2b50 0x7f2978151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f297cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f297815c680 0x7f29780b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f297cc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f297815ff60 con 0x7f29780b6c70 2024-04-15T15:19:27.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f297cc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f297815ca60 con 0x7f297815c680 2024-04-15T15:19:27.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f297cc00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f29780d1770 con 0x7f29780d2b50 2024-04-15T15:19:27.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f2977400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29780b6c70 0x7f29780cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f2977400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29780b6c70 0x7f29780cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47408/0 (socket says 172.21.15.49:47408) 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f2977400640 1 -- 172.21.15.49:0/2584172130 learned_addr learned my addr 172.21.15.49:0/2584172130 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f2976a00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29780d2b50 0x7f2978151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f2977400640 1 -- 172.21.15.49:0/2584172130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29780d2b50 msgr2=0x7f2978151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f2977400640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29780d2b50 0x7f2978151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f2977400640 1 -- 172.21.15.49:0/2584172130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f297815c680 msgr2=0x7f29780b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.512+0000 7f2977e00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f297815c680 0x7f29780b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.513+0000 7f2977400640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f297815c680 0x7f29780b7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.513+0000 7f2977400640 1 -- 172.21.15.49:0/2584172130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29780cb170 con 0x7f29780b6c70 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.513+0000 7f2976a00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29780d2b50 0x7f2978151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.513+0000 7f2977e00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f297815c680 0x7f29780b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.513+0000 7f2977400640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29780b6c70 0x7f29780cab70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2968000c00 tx=0x7f29680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.513+0000 7f296f400640 1 -- 172.21.15.49:0/2584172130 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968040070 con 0x7f29780b6c70 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.513+0000 7f296f400640 1 -- 172.21.15.49:0/2584172130 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2968039a20 con 0x7f29780b6c70 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.513+0000 7f296f400640 1 -- 172.21.15.49:0/2584172130 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968035040 con 0x7f29780b6c70 2024-04-15T15:19:27.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.513+0000 7f297cc00640 1 -- 172.21.15.49:0/2584172130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29780cc9f0 con 0x7f29780b6c70 2024-04-15T15:19:27.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.513+0000 7f297cc00640 1 -- 172.21.15.49:0/2584172130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29780ccfb0 con 0x7f29780b6c70 2024-04-15T15:19:27.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.515+0000 7f297cc00640 1 -- 172.21.15.49:0/2584172130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29780755c0 con 0x7f29780b6c70 2024-04-15T15:19:27.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.516+0000 7f296f400640 1 -- 172.21.15.49:0/2584172130 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f296804b060 con 0x7f29780b6c70 2024-04-15T15:19:27.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.517+0000 7f296f400640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f295406c8f0 0x7f295406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.517+0000 7f296f400640 1 -- 172.21.15.49:0/2584172130 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f29680ad550 con 0x7f29780b6c70 2024-04-15T15:19:27.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.517+0000 7f2976a00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f295406c8f0 0x7f295406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.518+0000 7f2976a00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f295406c8f0 0x7f295406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f29780b3950 tx=0x7f2960009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.521+0000 7f296f400640 1 -- 172.21.15.49:0/2584172130 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f296807b980 con 0x7f29780b6c70 2024-04-15T15:19:27.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.728+0000 7f297cc00640 1 -- 172.21.15.49:0/2584172130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f297806dbe0 con 0x7f29780b6c70 2024-04-15T15:19:27.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.729+0000 7f296f400640 1 -- 172.21.15.49:0/2584172130 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f296807b280 con 0x7f29780b6c70 2024-04-15T15:19:27.731 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:27.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.732+0000 7f297cc00640 1 -- 172.21.15.49:0/2584172130 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f295406c8f0 msgr2=0x7f295406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.732+0000 7f297cc00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f295406c8f0 0x7f295406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f29780b3950 tx=0x7f2960009040 comp rx=0 tx=0).stop 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.732+0000 7f297cc00640 1 -- 172.21.15.49:0/2584172130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29780b6c70 msgr2=0x7f29780cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.732+0000 7f297cc00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29780b6c70 0x7f29780cab70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2968000c00 tx=0x7f29680046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.733+0000 7f297cc00640 1 -- 172.21.15.49:0/2584172130 shutdown_connections 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.733+0000 7f297cc00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f295406c8f0 0x7f295406edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.733+0000 7f297cc00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29780d2b50 0x7f2978151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.733+0000 7f297cc00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f297815c680 0x7f29780b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.733+0000 7f297cc00640 1 --2- 172.21.15.49:0/2584172130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29780b6c70 0x7f29780cab70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.733+0000 7f297cc00640 1 -- 172.21.15.49:0/2584172130 >> 172.21.15.49:0/2584172130 conn(0x7f2978073480 msgr2=0x7f29780d3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.733+0000 7f297cc00640 1 -- 172.21.15.49:0/2584172130 shutdown_connections 2024-04-15T15:19:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.733+0000 7f297cc00640 1 -- 172.21.15.49:0/2584172130 wait complete. 2024-04-15T15:19:27.753 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:27.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.975+0000 7f67cd000640 1 Processor -- start 2024-04-15T15:19:27.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.975+0000 7f67cd000640 1 -- start start 2024-04-15T15:19:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.976+0000 7f67cd000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f67c815a440 0x7f67c815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67cd000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f67c806fc80 0x7f67c80cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67cd000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67c80cd620 0x7f67c8072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67cd000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f67c806e280 con 0x7f67c806fc80 2024-04-15T15:19:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67cd000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f67c815d520 con 0x7f67c815a440 2024-04-15T15:19:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67cd000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f67c815d800 con 0x7f67c80cd620 2024-04-15T15:19:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67c5400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f67c806fc80 0x7f67c80cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67c5400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f67c806fc80 0x7f67c80cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47410/0 (socket says 172.21.15.49:47410) 2024-04-15T15:19:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67c5400640 1 -- 172.21.15.49:0/1540579587 learned_addr learned my addr 172.21.15.49:0/1540579587 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67c5e00640 1 --2- 172.21.15.49:0/1540579587 >> v2:172.21.15.107:3300/0 conn(0x7f67c815a440 0x7f67c815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67c6800640 1 --2- 172.21.15.49:0/1540579587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67c80cd620 0x7f67c8072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.977+0000 7f67c5400640 1 -- 172.21.15.49:0/1540579587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67c80cd620 msgr2=0x7f67c8072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.978+0000 7f67c5400640 1 --2- 172.21.15.49:0/1540579587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67c80cd620 0x7f67c8072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.978+0000 7f67c5400640 1 -- 172.21.15.49:0/1540579587 >> v2:172.21.15.107:3300/0 conn(0x7f67c815a440 msgr2=0x7f67c815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.978+0000 7f67c5400640 1 --2- 172.21.15.49:0/1540579587 >> v2:172.21.15.107:3300/0 conn(0x7f67c815a440 0x7f67c815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.978+0000 7f67c5400640 1 -- 172.21.15.49:0/1540579587 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c815dae0 con 0x7f67c806fc80 2024-04-15T15:19:27.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.979+0000 7f67c5400640 1 --2- 172.21.15.49:0/1540579587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c806fc80 0x7f67c80cd0e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f67c806c9a0 tx=0x7f67b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95ae7dd78f0196f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.979+0000 7f67c4a00640 1 -- 172.21.15.49:0/1540579587 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4035020 con 0x7f67c806fc80 2024-04-15T15:19:27.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.979+0000 7f67c4a00640 1 -- 172.21.15.49:0/1540579587 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b4031870 con 0x7f67c806fc80 2024-04-15T15:19:27.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.979+0000 7f67c4a00640 1 -- 172.21.15.49:0/1540579587 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4031c40 con 0x7f67c806fc80 2024-04-15T15:19:27.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.980+0000 7f67cd000640 1 -- 172.21.15.49:0/1540579587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c806fc80 msgr2=0x7f67c80cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.980+0000 7f67cd000640 1 --2- 172.21.15.49:0/1540579587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c806fc80 0x7f67c80cd0e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f67c806c9a0 tx=0x7f67b402b910 comp rx=0 tx=0).stop 2024-04-15T15:19:27.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.980+0000 7f67cd000640 1 -- 172.21.15.49:0/1540579587 shutdown_connections 2024-04-15T15:19:27.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.980+0000 7f67cd000640 1 --2- 172.21.15.49:0/1540579587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67c80cd620 0x7f67c8072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.980+0000 7f67cd000640 1 --2- 172.21.15.49:0/1540579587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c806fc80 0x7f67c80cd0e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.980+0000 7f67cd000640 1 --2- 172.21.15.49:0/1540579587 >> v2:172.21.15.107:3300/0 conn(0x7f67c815a440 0x7f67c815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.980+0000 7f67cd000640 1 -- 172.21.15.49:0/1540579587 >> 172.21.15.49:0/1540579587 conn(0x7f67c80b2460 msgr2=0x7f67c80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:27.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.980+0000 7f67cd000640 1 -- 172.21.15.49:0/1540579587 shutdown_connections 2024-04-15T15:19:27.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.980+0000 7f67cd000640 1 -- 172.21.15.49:0/1540579587 wait complete. 2024-04-15T15:19:27.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.982+0000 7f67cd000640 1 Processor -- start 2024-04-15T15:19:27.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.982+0000 7f67cd000640 1 -- start start 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.983+0000 7f67cd000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67c806fc80 0x7f67c80ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.983+0000 7f67cd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67c80cd620 0x7f67c81f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.983+0000 7f67cd000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c815a440 0x7f67c80aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.983+0000 7f67cd000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f67c806f510 con 0x7f67c815a440 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.983+0000 7f67cd000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f67c815ddc0 con 0x7f67c80cd620 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.983+0000 7f67cd000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f67c8072690 con 0x7f67c806fc80 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c815a440 0x7f67c80aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c815a440 0x7f67c80aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47412/0 (socket says 172.21.15.49:47412) 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c6800640 1 -- 172.21.15.49:0/959111094 learned_addr learned my addr 172.21.15.49:0/959111094 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c5400640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67c80cd620 0x7f67c81f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c5e00640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67c806fc80 0x7f67c80ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c6800640 1 -- 172.21.15.49:0/959111094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67c806fc80 msgr2=0x7f67c80ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c6800640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67c806fc80 0x7f67c80ad3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c6800640 1 -- 172.21.15.49:0/959111094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67c80cd620 msgr2=0x7f67c81f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:27.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c6800640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67c80cd620 0x7f67c81f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:27.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c6800640 1 -- 172.21.15.49:0/959111094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c8145170 con 0x7f67c815a440 2024-04-15T15:19:27.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.984+0000 7f67c5400640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67c80cd620 0x7f67c81f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.985+0000 7f67c6800640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c815a440 0x7f67c80aee50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007e30 tx=0x7f67c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.985+0000 7f67be000640 1 -- 172.21.15.49:0/959111094 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0014420 con 0x7f67c815a440 2024-04-15T15:19:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.985+0000 7f67be000640 1 -- 172.21.15.49:0/959111094 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c0019070 con 0x7f67c815a440 2024-04-15T15:19:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.985+0000 7f67be000640 1 -- 172.21.15.49:0/959111094 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0014800 con 0x7f67c815a440 2024-04-15T15:19:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.985+0000 7f67cd000640 1 -- 172.21.15.49:0/959111094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c80addb0 con 0x7f67c815a440 2024-04-15T15:19:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.985+0000 7f67cd000640 1 -- 172.21.15.49:0/959111094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c80ae370 con 0x7f67c815a440 2024-04-15T15:19:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.986+0000 7f67cd000640 1 -- 172.21.15.49:0/959111094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c8068c80 con 0x7f67c815a440 2024-04-15T15:19:27.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.988+0000 7f67be000640 1 -- 172.21.15.49:0/959111094 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67c00149a0 con 0x7f67c815a440 2024-04-15T15:19:27.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.989+0000 7f67be000640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f67a806c8f0 0x7f67a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:27.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.989+0000 7f67be000640 1 -- 172.21.15.49:0/959111094 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f67c008c390 con 0x7f67c815a440 2024-04-15T15:19:27.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.989+0000 7f67c5e00640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f67a806c8f0 0x7f67a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:27.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.990+0000 7f67c5e00640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f67a806c8f0 0x7f67a806edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f67b8006fd0 tx=0x7f67b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:27.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:27.993+0000 7f67be000640 1 -- 172.21.15.49:0/959111094 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67c005a7b0 con 0x7f67c815a440 2024-04-15T15:19:28.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.197+0000 7f67cd000640 1 -- 172.21.15.49:0/959111094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f67c806d130 con 0x7f67c815a440 2024-04-15T15:19:28.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.198+0000 7f67be000640 1 -- 172.21.15.49:0/959111094 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f67c005a0b0 con 0x7f67c815a440 2024-04-15T15:19:28.200 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:28.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.200+0000 7f67cd000640 1 -- 172.21.15.49:0/959111094 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f67a806c8f0 msgr2=0x7f67a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.200+0000 7f67cd000640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f67a806c8f0 0x7f67a806edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f67b8006fd0 tx=0x7f67b8008040 comp rx=0 tx=0).stop 2024-04-15T15:19:28.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.201+0000 7f67cd000640 1 -- 172.21.15.49:0/959111094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c815a440 msgr2=0x7f67c80aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.201+0000 7f67cd000640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c815a440 0x7f67c80aee50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007e30 tx=0x7f67c000e4b0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.201+0000 7f67cd000640 1 -- 172.21.15.49:0/959111094 shutdown_connections 2024-04-15T15:19:28.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.201+0000 7f67cd000640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f67a806c8f0 0x7f67a806edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.201+0000 7f67cd000640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67c815a440 0x7f67c80aee50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.201+0000 7f67cd000640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67c80cd620 0x7f67c81f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.201+0000 7f67cd000640 1 --2- 172.21.15.49:0/959111094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67c806fc80 0x7f67c80ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.201+0000 7f67cd000640 1 -- 172.21.15.49:0/959111094 >> 172.21.15.49:0/959111094 conn(0x7f67c80b2460 msgr2=0x7f67c815f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:28.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.201+0000 7f67cd000640 1 -- 172.21.15.49:0/959111094 shutdown_connections 2024-04-15T15:19:28.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.201+0000 7f67cd000640 1 -- 172.21.15.49:0/959111094 wait complete. 2024-04-15T15:19:28.233 INFO:tasks.cephfs_test_runner:Starting test: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) That a file with ceph.quiesce.block is not quiesced. 2024-04-15T15:19:28.233 INFO:tasks.cephfs_test_runner:test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:19:28.234 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:28.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.463+0000 7fe22dc00640 1 Processor -- start 2024-04-15T15:19:28.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.464+0000 7fe22dc00640 1 -- start start 2024-04-15T15:19:28.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.465+0000 7fe22dc00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe228162020 0x7fe228162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.465+0000 7fe22dc00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe2280d1c90 0x7fe2280d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.465+0000 7fe22dc00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2280d0090 con 0x7fe2280cf110 2024-04-15T15:19:28.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.465+0000 7fe22dc00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fe228163440 con 0x7fe2280d1c90 2024-04-15T15:19:28.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.465+0000 7fe22dc00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe2280b7b30 con 0x7fe228162020 2024-04-15T15:19:28.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.466+0000 7fe227e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe228162020 0x7fe228162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.466+0000 7fe227e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe228162020 0x7fe228162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:52182/0 (socket says 172.21.15.49:52182) 2024-04-15T15:19:28.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.466+0000 7fe227e00640 1 -- 172.21.15.49:0/1576540655 learned_addr learned my addr 172.21.15.49:0/1576540655 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:28.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.466+0000 7fe22d200640 1 --2- 172.21.15.49:0/1576540655 >> v2:172.21.15.107:3300/0 conn(0x7fe2280d1c90 0x7fe2280d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.467+0000 7fe227e00640 1 -- 172.21.15.49:0/1576540655 >> v2:172.21.15.107:3300/0 conn(0x7fe2280d1c90 msgr2=0x7fe2280d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:28.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.467+0000 7fe227e00640 1 --2- 172.21.15.49:0/1576540655 >> v2:172.21.15.107:3300/0 conn(0x7fe2280d1c90 0x7fe2280d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.467+0000 7fe227e00640 1 -- 172.21.15.49:0/1576540655 >> v1:172.21.15.107:6789/0 conn(0x7fe2280cf110 legacy=0x7fe2280d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:28.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.467+0000 7fe227e00640 1 -- 172.21.15.49:0/1576540655 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2280cfad0 con 0x7fe228162020 2024-04-15T15:19:28.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.468+0000 7fe227e00640 1 --2- 172.21.15.49:0/1576540655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe228162020 0x7fe228162f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe218009870 tx=0x7fe21802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b5b53962464b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.469+0000 7fe227400640 1 -- 172.21.15.49:0/1576540655 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218035020 con 0x7fe228162020 2024-04-15T15:19:28.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.469+0000 7fe227400640 1 -- 172.21.15.49:0/1576540655 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe218031640 con 0x7fe228162020 2024-04-15T15:19:28.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.469+0000 7fe227400640 1 -- 172.21.15.49:0/1576540655 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21802d040 con 0x7fe228162020 2024-04-15T15:19:28.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.470+0000 7fe22dc00640 1 -- 172.21.15.49:0/1576540655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe228162020 msgr2=0x7fe228162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.470+0000 7fe22dc00640 1 --2- 172.21.15.49:0/1576540655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe228162020 0x7fe228162f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe218009870 tx=0x7fe21802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.470+0000 7fe22dc00640 1 -- 172.21.15.49:0/1576540655 shutdown_connections 2024-04-15T15:19:28.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.470+0000 7fe22dc00640 1 --2- 172.21.15.49:0/1576540655 >> v2:172.21.15.107:3300/0 conn(0x7fe2280d1c90 0x7fe2280d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.470+0000 7fe22dc00640 1 --2- 172.21.15.49:0/1576540655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe228162020 0x7fe228162f00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.470+0000 7fe22dc00640 1 -- 172.21.15.49:0/1576540655 >> 172.21.15.49:0/1576540655 conn(0x7fe22806cf00 msgr2=0x7fe22806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:28.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.470+0000 7fe22dc00640 1 -- 172.21.15.49:0/1576540655 shutdown_connections 2024-04-15T15:19:28.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.470+0000 7fe22dc00640 1 -- 172.21.15.49:0/1576540655 wait complete. 2024-04-15T15:19:28.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.472+0000 7fe22dc00640 1 Processor -- start 2024-04-15T15:19:28.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.472+0000 7fe22dc00640 1 -- start start 2024-04-15T15:19:28.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe22dc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280cf110 0x7fe2281f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe22dc00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2280d1c90 0x7fe22814ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe22dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe228162020 0x7fe2280ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe22dc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe2280d2b90 con 0x7fe2280cf110 2024-04-15T15:19:28.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe22dc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2280cf880 con 0x7fe228162020 2024-04-15T15:19:28.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe22dc00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe2280d2e70 con 0x7fe2280d1c90 2024-04-15T15:19:28.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe227e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280cf110 0x7fe2281f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe227e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280cf110 0x7fe2281f5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47422/0 (socket says 172.21.15.49:47422) 2024-04-15T15:19:28.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe227e00640 1 -- 172.21.15.49:0/2139686589 learned_addr learned my addr 172.21.15.49:0/2139686589 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:28.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe22d200640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe228162020 0x7fe2280ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.474+0000 7fe21fe00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2280d1c90 0x7fe22814ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.475+0000 7fe227e00640 1 -- 172.21.15.49:0/2139686589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2280d1c90 msgr2=0x7fe22814ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.475+0000 7fe227e00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2280d1c90 0x7fe22814ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.475+0000 7fe227e00640 1 -- 172.21.15.49:0/2139686589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe228162020 msgr2=0x7fe2280ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.475+0000 7fe227e00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe228162020 0x7fe2280ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.475+0000 7fe227e00640 1 -- 172.21.15.49:0/2139686589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2281504d0 con 0x7fe2280cf110 2024-04-15T15:19:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.475+0000 7fe227e00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280cf110 0x7fe2281f5fd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe218000c00 tx=0x7fe2180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.475+0000 7fe224c00640 1 -- 172.21.15.49:0/2139686589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21803aa60 con 0x7fe2280cf110 2024-04-15T15:19:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.475+0000 7fe224c00640 1 -- 172.21.15.49:0/2139686589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21803ac20 con 0x7fe2280cf110 2024-04-15T15:19:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.475+0000 7fe224c00640 1 -- 172.21.15.49:0/2139686589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2180443f0 con 0x7fe2280cf110 2024-04-15T15:19:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.475+0000 7fe22dc00640 1 -- 172.21.15.49:0/2139686589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2281f5c40 con 0x7fe2280cf110 2024-04-15T15:19:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.476+0000 7fe22dc00640 1 -- 172.21.15.49:0/2139686589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2280c8880 con 0x7fe2280cf110 2024-04-15T15:19:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.477+0000 7fe22dc00640 1 -- 172.21.15.49:0/2139686589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2280cb8d0 con 0x7fe2280cf110 2024-04-15T15:19:28.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.478+0000 7fe224c00640 1 -- 172.21.15.49:0/2139686589 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe218038020 con 0x7fe2280cf110 2024-04-15T15:19:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.479+0000 7fe224c00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe20406c8f0 0x7fe20406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.479+0000 7fe224c00640 1 -- 172.21.15.49:0/2139686589 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe2180ad7d0 con 0x7fe2280cf110 2024-04-15T15:19:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.479+0000 7fe21fe00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe20406c8f0 0x7fe20406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.480+0000 7fe21fe00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe20406c8f0 0x7fe20406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe210006fd0 tx=0x7fe210008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.484+0000 7fe224c00640 1 -- 172.21.15.49:0/2139686589 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe21803f070 con 0x7fe2280cf110 2024-04-15T15:19:28.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.690+0000 7fe22dc00640 1 -- 172.21.15.49:0/2139686589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2280b7650 con 0x7fe2280cf110 2024-04-15T15:19:28.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.693+0000 7fe224c00640 1 -- 172.21.15.49:0/2139686589 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7fe2280d0090 con 0x7fe2280cf110 2024-04-15T15:19:28.695 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:28.695 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:18:46.807736+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:28.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.695+0000 7fe22dc00640 1 -- 172.21.15.49:0/2139686589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe20406c8f0 msgr2=0x7fe20406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.695+0000 7fe22dc00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe20406c8f0 0x7fe20406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe210006fd0 tx=0x7fe210008040 comp rx=0 tx=0).stop 2024-04-15T15:19:28.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.695+0000 7fe22dc00640 1 -- 172.21.15.49:0/2139686589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280cf110 msgr2=0x7fe2281f5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.695+0000 7fe22dc00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280cf110 0x7fe2281f5fd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe218000c00 tx=0x7fe2180046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.695+0000 7fe22dc00640 1 -- 172.21.15.49:0/2139686589 shutdown_connections 2024-04-15T15:19:28.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.696+0000 7fe22dc00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe20406c8f0 0x7fe20406edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.696+0000 7fe22dc00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2280d1c90 0x7fe22814ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.696+0000 7fe22dc00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe228162020 0x7fe2280ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.696+0000 7fe22dc00640 1 --2- 172.21.15.49:0/2139686589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280cf110 0x7fe2281f5fd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.696+0000 7fe22dc00640 1 -- 172.21.15.49:0/2139686589 >> 172.21.15.49:0/2139686589 conn(0x7fe22806cf00 msgr2=0x7fe2281510e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:28.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.696+0000 7fe22dc00640 1 -- 172.21.15.49:0/2139686589 shutdown_connections 2024-04-15T15:19:28.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.696+0000 7fe22dc00640 1 -- 172.21.15.49:0/2139686589 wait complete. 2024-04-15T15:19:28.718 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:28.718 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:28.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.927+0000 7fd649e00640 1 Processor -- start 2024-04-15T15:19:28.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.927+0000 7fd649e00640 1 -- start start 2024-04-15T15:19:28.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.929+0000 7fd649e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd644147fc0 0x7fd6440b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.929+0000 7fd649e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6440b5e00 con 0x7fd644162b00 2024-04-15T15:19:28.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.929+0000 7fd649e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6441483a0 con 0x7fd6440aef00 2024-04-15T15:19:28.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.929+0000 7fd649e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd6440afaa0 con 0x7fd644147fc0 2024-04-15T15:19:28.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.929+0000 7fd649400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd644162b00 0x7fd64406df00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45896/0 (socket says 172.21.15.49:45896) 2024-04-15T15:19:28.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.929+0000 7fd649400640 1 -- 172.21.15.49:0/3877849178 learned_addr learned my addr 172.21.15.49:0/3877849178 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd648a00640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd644147fc0 0x7fd6440b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2633512850 0 0) 0x7fd6440b5e00 con 0x7fd644162b00 2024-04-15T15:19:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd62c003520 con 0x7fd644162b00 2024-04-15T15:19:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 66149096 0 0) 0x7fd62c003520 con 0x7fd644162b00 2024-04-15T15:19:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd644147fc0 msgr2=0x7fd6440b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd644147fc0 0x7fd6440b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 >> v1:172.21.15.107:6789/0 conn(0x7fd6440aef00 legacy=0x7fd6441623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6440afd80 con 0x7fd644162b00 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fd63c0019f0 con 0x7fd644162b00 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 >> v1:172.21.15.49:6789/0 conn(0x7fd644162b00 legacy=0x7fd64406df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd62c0039e0 0x7fd62c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd62c004800 0x7fd62c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd644147fc0 0x7fd62c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd6440afaa0 con 0x7fd62c004800 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c00bb50 con 0x7fd62c0039e0 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.930+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd62c00bd50 con 0x7fd644147fc0 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.931+0000 7fd643e00640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd62c004800 0x7fd62c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.931+0000 7fd648a00640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd62c0039e0 0x7fd62c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.931+0000 7fd649400640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd644147fc0 0x7fd62c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.931+0000 7fd643e00640 1 -- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd644147fc0 msgr2=0x7fd62c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:28.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.931+0000 7fd643e00640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd644147fc0 0x7fd62c005270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.931+0000 7fd643e00640 1 -- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd62c0039e0 msgr2=0x7fd62c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:28.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.931+0000 7fd643e00640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd62c0039e0 0x7fd62c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.931+0000 7fd643e00640 1 -- 172.21.15.49:0/3877849178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62c003520 con 0x7fd62c004800 2024-04-15T15:19:28.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.931+0000 7fd648a00640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd62c0039e0 0x7fd62c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:28.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.932+0000 7fd643e00640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd62c004800 0x7fd62c004c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd630001750 tx=0x7fd62c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7d9b7898e0fb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.933+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630001370 con 0x7fd62c004800 2024-04-15T15:19:28.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.933+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6300052d0 con 0x7fd62c004800 2024-04-15T15:19:28.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.933+0000 7fd643400640 1 -- 172.21.15.49:0/3877849178 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630003830 con 0x7fd62c004800 2024-04-15T15:19:28.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.933+0000 7fd649e00640 1 -- 172.21.15.49:0/3877849178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd62c004800 msgr2=0x7fd62c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.933+0000 7fd649e00640 1 --2- 172.21.15.49:0/3877849178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd62c004800 0x7fd62c004c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd630001750 tx=0x7fd62c005830 comp rx=0 tx=0).stop 2024-04-15T15:19:28.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.933+0000 7fd649400640 1 -- 172.21.15.49:0/3877849178 reap_dead start 2024-04-15T15:19:28.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.933+0000 7fd649e00640 1 -- 172.21.15.49:0/3877849178 shutdown_connections 2024-04-15T15:19:28.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.933+0000 7fd649e00640 1 -- 172.21.15.49:0/3877849178 >> 172.21.15.49:0/3877849178 conn(0x7fd6440b23a0 msgr2=0x7fd6440b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:28.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.934+0000 7fd649e00640 1 -- 172.21.15.49:0/3877849178 shutdown_connections 2024-04-15T15:19:28.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.934+0000 7fd649e00640 1 -- 172.21.15.49:0/3877849178 wait complete. 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.935+0000 7fd649e00640 1 Processor -- start 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.936+0000 7fd649e00640 1 -- start start 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.936+0000 7fd649e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd62c004800 0x7fd644161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.936+0000 7fd649e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd64413e3f0 0x7fd6441f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.936+0000 7fd649e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6441f4d70 0x7fd64413f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.936+0000 7fd649e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd6440b6590 con 0x7fd6441f4d70 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.936+0000 7fd649e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6440b0060 con 0x7fd62c004800 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.936+0000 7fd649e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd6440b6710 con 0x7fd64413e3f0 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.937+0000 7fd649400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6441f4d70 0x7fd64413f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.937+0000 7fd649400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6441f4d70 0x7fd64413f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47440/0 (socket says 172.21.15.49:47440) 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.937+0000 7fd649400640 1 -- 172.21.15.49:0/2646462048 learned_addr learned my addr 172.21.15.49:0/2646462048 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.937+0000 7fd648a00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd62c004800 0x7fd644161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.937+0000 7fd643e00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd64413e3f0 0x7fd6441f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.937+0000 7fd649400640 1 -- 172.21.15.49:0/2646462048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd64413e3f0 msgr2=0x7fd6441f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.937+0000 7fd649400640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd64413e3f0 0x7fd6441f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.937+0000 7fd649400640 1 -- 172.21.15.49:0/2646462048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd62c004800 msgr2=0x7fd644161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.937+0000 7fd649400640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd62c004800 0x7fd644161580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.937+0000 7fd649400640 1 -- 172.21.15.49:0/2646462048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64413e8d0 con 0x7fd6441f4d70 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.938+0000 7fd649400640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6441f4d70 0x7fd64413f970 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd6440af5f0 tx=0x7fd63c00b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.938+0000 7fd640c00640 1 -- 172.21.15.49:0/2646462048 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00bad0 con 0x7fd6441f4d70 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.938+0000 7fd640c00640 1 -- 172.21.15.49:0/2646462048 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63c00bc70 con 0x7fd6441f4d70 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.938+0000 7fd640c00640 1 -- 172.21.15.49:0/2646462048 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c004e30 con 0x7fd6441f4d70 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.938+0000 7fd649e00640 1 -- 172.21.15.49:0/2646462048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64413ebb0 con 0x7fd6441f4d70 2024-04-15T15:19:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.938+0000 7fd649e00640 1 -- 172.21.15.49:0/2646462048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64413f170 con 0x7fd6441f4d70 2024-04-15T15:19:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.939+0000 7fd649e00640 1 -- 172.21.15.49:0/2646462048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6440755c0 con 0x7fd6441f4d70 2024-04-15T15:19:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.940+0000 7fd640c00640 1 -- 172.21.15.49:0/2646462048 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd63c01b020 con 0x7fd6441f4d70 2024-04-15T15:19:28.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.942+0000 7fd640c00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd62406c8f0 0x7fd62406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:28.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.942+0000 7fd640c00640 1 -- 172.21.15.49:0/2646462048 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd63c08bd40 con 0x7fd6441f4d70 2024-04-15T15:19:28.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.942+0000 7fd648a00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd62406c8f0 0x7fd62406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:28.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.943+0000 7fd648a00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd62406c8f0 0x7fd62406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd638009aa0 tx=0x7fd638008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:28.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:28.946+0000 7fd640c00640 1 -- 172.21.15.49:0/2646462048 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd63c05a160 con 0x7fd6441f4d70 2024-04-15T15:19:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.155+0000 7fd649e00640 1 -- 172.21.15.49:0/2646462048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd64406ff20 con 0x7fd6441f4d70 2024-04-15T15:19:29.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.156+0000 7fd640c00640 1 -- 172.21.15.49:0/2646462048 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fd63c059a60 con 0x7fd6441f4d70 2024-04-15T15:19:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 -- 172.21.15.49:0/2646462048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd62406c8f0 msgr2=0x7fd62406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd62406c8f0 0x7fd62406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd638009aa0 tx=0x7fd638008040 comp rx=0 tx=0).stop 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 -- 172.21.15.49:0/2646462048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6441f4d70 msgr2=0x7fd64413f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6441f4d70 0x7fd64413f970 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd6440af5f0 tx=0x7fd63c00b0d0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 -- 172.21.15.49:0/2646462048 shutdown_connections 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd62406c8f0 0x7fd62406edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd64413e3f0 0x7fd6441f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6441f4d70 0x7fd64413f970 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 --2- 172.21.15.49:0/2646462048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd62c004800 0x7fd644161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 -- 172.21.15.49:0/2646462048 >> 172.21.15.49:0/2646462048 conn(0x7fd6440b23a0 msgr2=0x7fd6441623f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:29.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 -- 172.21.15.49:0/2646462048 shutdown_connections 2024-04-15T15:19:29.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.159+0000 7fd649e00640 1 -- 172.21.15.49:0/2646462048 wait complete. 2024-04-15T15:19:29.181 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:29.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.401+0000 7f8264a00640 1 Processor -- start 2024-04-15T15:19:29.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.402+0000 7f8264a00640 1 -- start start 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.403+0000 7f8264a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f82601461c0 0x7f8260071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.403+0000 7f8264a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8260072420 0x7f82600ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.403+0000 7f8264a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 0x7f8260160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.403+0000 7f8264a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f82600cf1d0 con 0x7f82601461c0 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.403+0000 7f8264a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f8260072880 con 0x7f8260072420 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.403+0000 7f8264a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f82600cd080 con 0x7f82600ce9e0 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f825f400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f82601461c0 0x7f8260071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f825f400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f82601461c0 0x7f8260071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47444/0 (socket says 172.21.15.49:47444) 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f825f400640 1 -- 172.21.15.49:0/273776132 learned_addr learned my addr 172.21.15.49:0/273776132 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f825fe00640 1 --2- 172.21.15.49:0/273776132 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 0x7f8260160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f8257e00640 1 --2- 172.21.15.49:0/273776132 >> v2:172.21.15.107:3300/0 conn(0x7f8260072420 0x7f82600ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f825f400640 1 -- 172.21.15.49:0/273776132 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 msgr2=0x7f8260160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f825f400640 1 --2- 172.21.15.49:0/273776132 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 0x7f8260160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f825f400640 1 -- 172.21.15.49:0/273776132 >> v2:172.21.15.107:3300/0 conn(0x7f8260072420 msgr2=0x7f82600ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f825f400640 1 --2- 172.21.15.49:0/273776132 >> v2:172.21.15.107:3300/0 conn(0x7f8260072420 0x7f82600ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f825f400640 1 -- 172.21.15.49:0/273776132 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82600cd2d0 con 0x7f82601461c0 2024-04-15T15:19:29.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f8257e00640 1 --2- 172.21.15.49:0/273776132 >> v2:172.21.15.107:3300/0 conn(0x7f8260072420 0x7f82600ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:29.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.404+0000 7f825fe00640 1 --2- 172.21.15.49:0/273776132 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 0x7f8260160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:29.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.406+0000 7f825f400640 1 --2- 172.21.15.49:0/273776132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 0x7f8260071ee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f82600737f0 tx=0x7f825002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd5828435824c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.406+0000 7f825ea00640 1 -- 172.21.15.49:0/273776132 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250035020 con 0x7f82601461c0 2024-04-15T15:19:29.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.406+0000 7f825ea00640 1 -- 172.21.15.49:0/273776132 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825002d040 con 0x7f82601461c0 2024-04-15T15:19:29.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.406+0000 7f825ea00640 1 -- 172.21.15.49:0/273776132 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250031a80 con 0x7f82601461c0 2024-04-15T15:19:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.407+0000 7f8264a00640 1 -- 172.21.15.49:0/273776132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 msgr2=0x7f8260071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.407+0000 7f8264a00640 1 --2- 172.21.15.49:0/273776132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 0x7f8260071ee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f82600737f0 tx=0x7f825002b900 comp rx=0 tx=0).stop 2024-04-15T15:19:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.407+0000 7f8264a00640 1 -- 172.21.15.49:0/273776132 shutdown_connections 2024-04-15T15:19:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.407+0000 7f8264a00640 1 --2- 172.21.15.49:0/273776132 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 0x7f8260160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.407+0000 7f8264a00640 1 --2- 172.21.15.49:0/273776132 >> v2:172.21.15.107:3300/0 conn(0x7f8260072420 0x7f82600ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.407+0000 7f8264a00640 1 --2- 172.21.15.49:0/273776132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 0x7f8260071ee0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.407+0000 7f8264a00640 1 -- 172.21.15.49:0/273776132 >> 172.21.15.49:0/273776132 conn(0x7f826006b930 msgr2=0x7f826006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.407+0000 7f8264a00640 1 -- 172.21.15.49:0/273776132 shutdown_connections 2024-04-15T15:19:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.407+0000 7f8264a00640 1 -- 172.21.15.49:0/273776132 wait complete. 2024-04-15T15:19:29.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.409+0000 7f8264a00640 1 Processor -- start 2024-04-15T15:19:29.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.409+0000 7f8264a00640 1 -- start start 2024-04-15T15:19:29.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f8264a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8260072420 0x7f826015a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f8264a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 0x7f82600ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f8264a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 0x7f82600ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f8264a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82600cdb70 con 0x7f82601461c0 2024-04-15T15:19:29.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f8264a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82600cd890 con 0x7f8260072420 2024-04-15T15:19:29.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f8264a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f82600cee70 con 0x7f82600ce9e0 2024-04-15T15:19:29.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f825fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 0x7f82600ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f825fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 0x7f82600ac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47448/0 (socket says 172.21.15.49:47448) 2024-04-15T15:19:29.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f825fe00640 1 -- 172.21.15.49:0/1971302601 learned_addr learned my addr 172.21.15.49:0/1971302601 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f825f400640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8260072420 0x7f826015a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.411+0000 7f8257e00640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 0x7f82600ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.412+0000 7f825fe00640 1 -- 172.21.15.49:0/1971302601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 msgr2=0x7f82600ac1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.412+0000 7f825fe00640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 0x7f82600ac1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.412+0000 7f825fe00640 1 -- 172.21.15.49:0/1971302601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8260072420 msgr2=0x7f826015a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.412+0000 7f825fe00640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8260072420 0x7f826015a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.412+0000 7f825fe00640 1 -- 172.21.15.49:0/1971302601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826015ab60 con 0x7f82601461c0 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.412+0000 7f825fe00640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 0x7f82600ac710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8258009fa0 tx=0x7f825800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.412+0000 7f8257400640 1 -- 172.21.15.49:0/1971302601 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825801a070 con 0x7f82601461c0 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.412+0000 7f8257400640 1 -- 172.21.15.49:0/1971302601 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8258003bb0 con 0x7f82601461c0 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.412+0000 7f8264a00640 1 -- 172.21.15.49:0/1971302601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826015ae40 con 0x7f82601461c0 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.412+0000 7f8257400640 1 -- 172.21.15.49:0/1971302601 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258004030 con 0x7f82601461c0 2024-04-15T15:19:29.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.413+0000 7f8264a00640 1 -- 172.21.15.49:0/1971302601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826015b400 con 0x7f82601461c0 2024-04-15T15:19:29.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.414+0000 7f8264a00640 1 -- 172.21.15.49:0/1971302601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82600755c0 con 0x7f82601461c0 2024-04-15T15:19:29.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.415+0000 7f8257400640 1 -- 172.21.15.49:0/1971302601 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f825801d070 con 0x7f82601461c0 2024-04-15T15:19:29.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.416+0000 7f8257400640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f823c06c920 0x7f823c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.416+0000 7f8257400640 1 -- 172.21.15.49:0/1971302601 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f825808be20 con 0x7f82601461c0 2024-04-15T15:19:29.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.416+0000 7f825f400640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f823c06c920 0x7f823c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.417+0000 7f825f400640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f823c06c920 0x7f823c06ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8250036000 tx=0x7f825003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.420+0000 7f8257400640 1 -- 172.21.15.49:0/1971302601 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8258010070 con 0x7f82601461c0 2024-04-15T15:19:29.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.626+0000 7f8264a00640 1 -- 172.21.15.49:0/1971302601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8260072e40 con 0x7f82601461c0 2024-04-15T15:19:29.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.627+0000 7f8257400640 1 -- 172.21.15.49:0/1971302601 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f825805a1c0 con 0x7f82601461c0 2024-04-15T15:19:29.629 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:29.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.629+0000 7f8264a00640 1 -- 172.21.15.49:0/1971302601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f823c06c920 msgr2=0x7f823c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.629+0000 7f8264a00640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f823c06c920 0x7f823c06ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8250036000 tx=0x7f825003c040 comp rx=0 tx=0).stop 2024-04-15T15:19:29.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.629+0000 7f8264a00640 1 -- 172.21.15.49:0/1971302601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 msgr2=0x7f82600ac710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.629+0000 7f8264a00640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 0x7f82600ac710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8258009fa0 tx=0x7f825800d560 comp rx=0 tx=0).stop 2024-04-15T15:19:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.630+0000 7f8264a00640 1 -- 172.21.15.49:0/1971302601 shutdown_connections 2024-04-15T15:19:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.630+0000 7f8264a00640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f823c06c920 0x7f823c06ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.630+0000 7f8264a00640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82600ce9e0 0x7f82600ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.630+0000 7f8264a00640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82601461c0 0x7f82600ac710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.630+0000 7f8264a00640 1 --2- 172.21.15.49:0/1971302601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8260072420 0x7f826015a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.630+0000 7f8264a00640 1 -- 172.21.15.49:0/1971302601 >> 172.21.15.49:0/1971302601 conn(0x7f826006b930 msgr2=0x7f82600b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.630+0000 7f8264a00640 1 -- 172.21.15.49:0/1971302601 shutdown_connections 2024-04-15T15:19:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.630+0000 7f8264a00640 1 -- 172.21.15.49:0/1971302601 wait complete. 2024-04-15T15:19:29.652 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:29.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.882+0000 7f5907200640 1 Processor -- start 2024-04-15T15:19:29.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.882+0000 7f5907200640 1 -- start start 2024-04-15T15:19:29.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.884+0000 7f5907200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5908158f10 0x7f590806f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.884+0000 7f5907200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f590806f570 0x7f59080cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.884+0000 7f5907200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f590815f230 con 0x7f59080b3dc0 2024-04-15T15:19:29.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.884+0000 7f5907200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f590806f9c0 con 0x7f590806f570 2024-04-15T15:19:29.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.884+0000 7f5907200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f590815f4d0 con 0x7f5908158f10 2024-04-15T15:19:29.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.884+0000 7f58fd400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f59080b3dc0 0x7f59080cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45912/0 (socket says 172.21.15.49:45912) 2024-04-15T15:19:29.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.884+0000 7f58fd400640 1 -- 172.21.15.49:0/3736182696 learned_addr learned my addr 172.21.15.49:0/3736182696 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:29.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5906800640 1 --2- 172.21.15.49:0/3736182696 >> v2:172.21.15.107:3300/0 conn(0x7f590806f570 0x7f59080cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905e00640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5908158f10 0x7f590806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 699013854 0 0) 0x7f590815f230 con 0x7f59080b3dc0 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58ec003520 con 0x7f59080b3dc0 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3100249735 0 0) 0x7f58ec003520 con 0x7f59080b3dc0 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5908158f10 msgr2=0x7f590806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5908158f10 0x7f590806f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 >> v2:172.21.15.107:3300/0 conn(0x7f590806f570 msgr2=0x7f59080cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 --2- 172.21.15.49:0/3736182696 >> v2:172.21.15.107:3300/0 conn(0x7f590806f570 0x7f59080cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590815f7b0 con 0x7f59080b3dc0 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f58f0001690 con 0x7f59080b3dc0 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 >> v1:172.21.15.49:6789/0 conn(0x7f59080b3dc0 legacy=0x7f59080cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.885+0000 7f5905400640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5908158f10 0x7f58ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5905400640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec003f80 0x7f58ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5905400640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f590806f9c0 con 0x7f58ec004d40 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f590815f4d0 con 0x7f58ec003f80 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f58ec00dc80 con 0x7f5908158f10 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5906800640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f58fd400640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec003f80 0x7f58ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5905e00640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5908158f10 0x7f58ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5906800640 1 -- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5908158f10 msgr2=0x7f58ec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5906800640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5908158f10 0x7f58ec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5906800640 1 -- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec003f80 msgr2=0x7f58ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5906800640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec003f80 0x7f58ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5906800640 1 -- 172.21.15.49:0/3736182696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec003520 con 0x7f58ec004d40 2024-04-15T15:19:29.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.886+0000 7f5905e00640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5908158f10 0x7f58ec0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:29.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.887+0000 7f5906800640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5900009ac0 tx=0x7f58ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff704f834f2b1eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.888+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900039070 con 0x7f58ec004d40 2024-04-15T15:19:29.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.888+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f590002c710 con 0x7f58ec004d40 2024-04-15T15:19:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.888+0000 7f5905400640 1 -- 172.21.15.49:0/3736182696 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900009690 con 0x7f58ec004d40 2024-04-15T15:19:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.888+0000 7f5907200640 1 -- 172.21.15.49:0/3736182696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec004d40 msgr2=0x7f58ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.888+0000 7f5907200640 1 --2- 172.21.15.49:0/3736182696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5900009ac0 tx=0x7f58ec005830 comp rx=0 tx=0).stop 2024-04-15T15:19:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.888+0000 7f5906800640 1 -- 172.21.15.49:0/3736182696 reap_dead start 2024-04-15T15:19:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.888+0000 7f5907200640 1 -- 172.21.15.49:0/3736182696 shutdown_connections 2024-04-15T15:19:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.888+0000 7f5907200640 1 -- 172.21.15.49:0/3736182696 >> 172.21.15.49:0/3736182696 conn(0x7f590806b930 msgr2=0x7f590806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.889+0000 7f5907200640 1 -- 172.21.15.49:0/3736182696 shutdown_connections 2024-04-15T15:19:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.889+0000 7f5907200640 1 -- 172.21.15.49:0/3736182696 wait complete. 2024-04-15T15:19:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.890+0000 7f5907200640 1 Processor -- start 2024-04-15T15:19:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.891+0000 7f5907200640 1 -- start start 2024-04-15T15:19:29.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.891+0000 7f5907200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec004d40 0x7f59081f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.891+0000 7f5907200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590815cc40 0x7f590815d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.891+0000 7f5907200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59080cbd30 0x7f59080cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5907200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f59081592f0 con 0x7f59080cbd30 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5907200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f590815fd70 con 0x7f58ec004d40 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5907200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f5908159470 con 0x7f590815cc40 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5906800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59080cbd30 0x7f59080cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5906800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59080cbd30 0x7f59080cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47462/0 (socket says 172.21.15.49:47462) 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5906800640 1 -- 172.21.15.49:0/1553326494 learned_addr learned my addr 172.21.15.49:0/1553326494 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f58fd400640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590815cc40 0x7f590815d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5905e00640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec004d40 0x7f59081f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5906800640 1 -- 172.21.15.49:0/1553326494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590815cc40 msgr2=0x7f590815d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5906800640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590815cc40 0x7f590815d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5906800640 1 -- 172.21.15.49:0/1553326494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec004d40 msgr2=0x7f59081f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5906800640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec004d40 0x7f59081f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.892+0000 7f5906800640 1 -- 172.21.15.49:0/1553326494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5908143cf0 con 0x7f59080cbd30 2024-04-15T15:19:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.893+0000 7f5906800640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59080cbd30 0x7f59080cc1b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5900009ac0 tx=0x7f5900004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.893+0000 7f58fea00640 1 -- 172.21.15.49:0/1553326494 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900039070 con 0x7f59080cbd30 2024-04-15T15:19:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.893+0000 7f58fea00640 1 -- 172.21.15.49:0/1553326494 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f590003a040 con 0x7f59080cbd30 2024-04-15T15:19:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.893+0000 7f58fea00640 1 -- 172.21.15.49:0/1553326494 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900003610 con 0x7f59080cbd30 2024-04-15T15:19:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.893+0000 7f5907200640 1 -- 172.21.15.49:0/1553326494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5908143fd0 con 0x7f59080cbd30 2024-04-15T15:19:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.893+0000 7f5907200640 1 -- 172.21.15.49:0/1553326494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5908144590 con 0x7f59080cbd30 2024-04-15T15:19:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.894+0000 7f5907200640 1 -- 172.21.15.49:0/1553326494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59080755c0 con 0x7f59080cbd30 2024-04-15T15:19:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.895+0000 7f58fea00640 1 -- 172.21.15.49:0/1553326494 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59000037b0 con 0x7f59080cbd30 2024-04-15T15:19:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.896+0000 7f58fea00640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f58e406c920 0x7f58e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.896+0000 7f58fea00640 1 -- 172.21.15.49:0/1553326494 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f59000acfd0 con 0x7f59080cbd30 2024-04-15T15:19:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.896+0000 7f5905e00640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f58e406c920 0x7f58e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:29.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.897+0000 7f5905e00640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f58e406c920 0x7f58e406ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f58f8003be0 tx=0x7f58f800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:29.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:29.900+0000 7f58fea00640 1 -- 172.21.15.49:0/1553326494 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f590007b3f0 con 0x7f59080cbd30 2024-04-15T15:19:30.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.107+0000 7f5907200640 1 -- 172.21.15.49:0/1553326494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f59080b38e0 con 0x7f59080cbd30 2024-04-15T15:19:30.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.108+0000 7f58fea00640 1 -- 172.21.15.49:0/1553326494 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f590007acf0 con 0x7f59080cbd30 2024-04-15T15:19:30.110 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 -- 172.21.15.49:0/1553326494 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f58e406c920 msgr2=0x7f58e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f58e406c920 0x7f58e406ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f58f8003be0 tx=0x7f58f800b040 comp rx=0 tx=0).stop 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 -- 172.21.15.49:0/1553326494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59080cbd30 msgr2=0x7f59080cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59080cbd30 0x7f59080cc1b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5900009ac0 tx=0x7f5900004730 comp rx=0 tx=0).stop 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 -- 172.21.15.49:0/1553326494 shutdown_connections 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f58e406c920 0x7f58e406ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590815cc40 0x7f590815d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59080cbd30 0x7f59080cc1b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 --2- 172.21.15.49:0/1553326494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec004d40 0x7f59081f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 -- 172.21.15.49:0/1553326494 >> 172.21.15.49:0/1553326494 conn(0x7f590806b930 msgr2=0x7f59080b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 -- 172.21.15.49:0/1553326494 shutdown_connections 2024-04-15T15:19:30.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.111+0000 7f5907200640 1 -- 172.21.15.49:0/1553326494 wait complete. 2024-04-15T15:19:30.133 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked' 2024-04-15T15:19:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.361+0000 7f44de200640 1 Processor -- start 2024-04-15T15:19:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.362+0000 7f44de200640 1 -- start start 2024-04-15T15:19:30.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.363+0000 7f44de200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44d8148130 0x7f44d815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.363+0000 7f44de200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f44d8162380 0x7f44d8071f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.363+0000 7f44de200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d8162940 con 0x7f44d8072490 2024-04-15T15:19:30.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.363+0000 7f44de200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f44d8072900 con 0x7f44d8148130 2024-04-15T15:19:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.363+0000 7f44de200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f44d8148510 con 0x7f44d8162380 2024-04-15T15:19:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.364+0000 7f44dce00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44d8148130 0x7f44d815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.364+0000 7f44dd800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f44d8072490 0x7f44d806dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45918/0 (socket says 172.21.15.49:45918) 2024-04-15T15:19:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.364+0000 7f44dce00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44d8148130 0x7f44d815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47466/0 (socket says 172.21.15.49:47466) 2024-04-15T15:19:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.364+0000 7f44dd800640 1 -- 172.21.15.49:0/273398290 learned_addr learned my addr 172.21.15.49:0/273398290 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.364+0000 7f44d7e00640 1 --2- 172.21.15.49:0/273398290 >> v2:172.21.15.107:3300/0 conn(0x7f44d8162380 0x7f44d8071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.364+0000 7f44dce00640 1 -- 172.21.15.49:0/273398290 >> v2:172.21.15.107:3300/0 conn(0x7f44d8162380 msgr2=0x7f44d8071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:30.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.364+0000 7f44dce00640 1 --2- 172.21.15.49:0/273398290 >> v2:172.21.15.107:3300/0 conn(0x7f44d8162380 0x7f44d8071f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.364+0000 7f44dce00640 1 -- 172.21.15.49:0/273398290 >> v1:172.21.15.49:6789/0 conn(0x7f44d8072490 legacy=0x7f44d806dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.364+0000 7f44dce00640 1 -- 172.21.15.49:0/273398290 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d8162c20 con 0x7f44d8148130 2024-04-15T15:19:30.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.364+0000 7f44d7400640 1 -- 172.21.15.49:0/273398290 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2720090344 0 0) 0x7f44d8162940 con 0x7f44d8072490 2024-04-15T15:19:30.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.366+0000 7f44dce00640 1 --2- 172.21.15.49:0/273398290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8148130 0x7f44d815f830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f44d80b3440 tx=0x7f44cc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368badc43e4c838d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.366+0000 7f44d7400640 1 -- 172.21.15.49:0/273398290 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0315a0 con 0x7f44d8148130 2024-04-15T15:19:30.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.366+0000 7f44d7400640 1 -- 172.21.15.49:0/273398290 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44cc037070 con 0x7f44d8148130 2024-04-15T15:19:30.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.366+0000 7f44d7400640 1 -- 172.21.15.49:0/273398290 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc031960 con 0x7f44d8148130 2024-04-15T15:19:30.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.366+0000 7f44de200640 1 -- 172.21.15.49:0/273398290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8148130 msgr2=0x7f44d815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.366+0000 7f44de200640 1 --2- 172.21.15.49:0/273398290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8148130 0x7f44d815f830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f44d80b3440 tx=0x7f44cc02b840 comp rx=0 tx=0).stop 2024-04-15T15:19:30.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.367+0000 7f44de200640 1 -- 172.21.15.49:0/273398290 shutdown_connections 2024-04-15T15:19:30.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.367+0000 7f44de200640 1 --2- 172.21.15.49:0/273398290 >> v2:172.21.15.107:3300/0 conn(0x7f44d8162380 0x7f44d8071f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.367+0000 7f44de200640 1 --2- 172.21.15.49:0/273398290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8148130 0x7f44d815f830 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.367+0000 7f44de200640 1 -- 172.21.15.49:0/273398290 >> 172.21.15.49:0/273398290 conn(0x7f44d806b930 msgr2=0x7f44d806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:30.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.367+0000 7f44de200640 1 -- 172.21.15.49:0/273398290 shutdown_connections 2024-04-15T15:19:30.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.367+0000 7f44de200640 1 -- 172.21.15.49:0/273398290 wait complete. 2024-04-15T15:19:30.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.368+0000 7f44de200640 1 Processor -- start 2024-04-15T15:19:30.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.369+0000 7f44de200640 1 -- start start 2024-04-15T15:19:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.369+0000 7f44de200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44d8072490 0x7f44d80cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.369+0000 7f44de200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d8148130 0x7f44d80b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.369+0000 7f44de200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8162380 0x7f44d815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44de200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44d81631e0 con 0x7f44d8162380 2024-04-15T15:19:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44de200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44d815fd70 con 0x7f44d8148130 2024-04-15T15:19:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44de200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f44d80aeb90 con 0x7f44d8072490 2024-04-15T15:19:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44dd800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8162380 0x7f44d815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44dd800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8162380 0x7f44d815e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47482/0 (socket says 172.21.15.49:47482) 2024-04-15T15:19:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44dd800640 1 -- 172.21.15.49:0/3203638635 learned_addr learned my addr 172.21.15.49:0/3203638635 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44dce00640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44d8072490 0x7f44d80cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44d7e00640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d8148130 0x7f44d80b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44dd800640 1 -- 172.21.15.49:0/3203638635 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44d8072490 msgr2=0x7f44d80cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44dd800640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44d8072490 0x7f44d80cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44dd800640 1 -- 172.21.15.49:0/3203638635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d8148130 msgr2=0x7f44d80b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:30.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44dd800640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d8148130 0x7f44d80b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:30.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.370+0000 7f44dd800640 1 -- 172.21.15.49:0/3203638635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d80b0bf0 con 0x7f44d8162380 2024-04-15T15:19:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.371+0000 7f44dd800640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8162380 0x7f44d815e070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44d000c730 tx=0x7f44d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.371+0000 7f44d4c00640 1 -- 172.21.15.49:0/3203638635 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0018070 con 0x7f44d8162380 2024-04-15T15:19:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.371+0000 7f44d4c00640 1 -- 172.21.15.49:0/3203638635 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d000ce40 con 0x7f44d8162380 2024-04-15T15:19:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.371+0000 7f44de200640 1 -- 172.21.15.49:0/3203638635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d80b0ed0 con 0x7f44d8162380 2024-04-15T15:19:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.371+0000 7f44d4c00640 1 -- 172.21.15.49:0/3203638635 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d000a860 con 0x7f44d8162380 2024-04-15T15:19:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.371+0000 7f44de200640 1 -- 172.21.15.49:0/3203638635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d80b1490 con 0x7f44d8162380 2024-04-15T15:19:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.372+0000 7f44de200640 1 -- 172.21.15.49:0/3203638635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d80755c0 con 0x7f44d8162380 2024-04-15T15:19:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.374+0000 7f44d4c00640 1 -- 172.21.15.49:0/3203638635 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44d001b070 con 0x7f44d8162380 2024-04-15T15:19:30.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.374+0000 7f44d4c00640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f44b806c8f0 0x7f44b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:30.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.375+0000 7f44d4c00640 1 -- 172.21.15.49:0/3203638635 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f44d008b150 con 0x7f44d8162380 2024-04-15T15:19:30.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.375+0000 7f44dce00640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f44b806c8f0 0x7f44b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:30.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.376+0000 7f44dce00640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f44b806c8f0 0x7f44b806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f44cc036000 tx=0x7f44cc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:30.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.379+0000 7f44d4c00640 1 -- 172.21.15.49:0/3203638635 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44d0059580 con 0x7f44d8162380 2024-04-15T15:19:30.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:30.662+0000 7f44de200640 1 -- 172.21.15.49:0/3203638635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]} v 0) -- 0x7f44d80adc10 con 0x7f44d8162380 2024-04-15T15:19:31.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.101+0000 7f44d4c00640 1 -- 172.21.15.49:0/3203638635 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]}]=0 v58) ==== 140+0+0 (secure 0 0 0) 0x7f44d0058e80 con 0x7f44d8162380 2024-04-15T15:19:31.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.107+0000 7f44de200640 1 -- 172.21.15.49:0/3203638635 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f44b806c8f0 msgr2=0x7f44b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.107+0000 7f44de200640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f44b806c8f0 0x7f44b806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f44cc036000 tx=0x7f44cc008030 comp rx=0 tx=0).stop 2024-04-15T15:19:31.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.107+0000 7f44de200640 1 -- 172.21.15.49:0/3203638635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8162380 msgr2=0x7f44d815e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.107+0000 7f44de200640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8162380 0x7f44d815e070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44d000c730 tx=0x7f44d000cbf0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.108+0000 7f44de200640 1 -- 172.21.15.49:0/3203638635 shutdown_connections 2024-04-15T15:19:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.108+0000 7f44de200640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f44b806c8f0 0x7f44b806edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.108+0000 7f44de200640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8162380 0x7f44d815e070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.108+0000 7f44de200640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d8148130 0x7f44d80b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.108+0000 7f44de200640 1 --2- 172.21.15.49:0/3203638635 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44d8072490 0x7f44d80cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.108+0000 7f44de200640 1 -- 172.21.15.49:0/3203638635 >> 172.21.15.49:0/3203638635 conn(0x7f44d806b930 msgr2=0x7f44d80b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.109+0000 7f44de200640 1 -- 172.21.15.49:0/3203638635 shutdown_connections 2024-04-15T15:19:31.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.110+0000 7f44de200640 1 -- 172.21.15.49:0/3203638635 wait complete. 2024-04-15T15:19:31.135 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.360+0000 7f9e5f200640 1 Processor -- start 2024-04-15T15:19:31.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.361+0000 7f9e5f200640 1 -- start start 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.362+0000 7f9e5f200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9e60162b80 0x7f9e600b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.362+0000 7f9e5f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e600b4390 0x7f9e60071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.362+0000 7f9e5f200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9e60162f60 0x7f9e6006f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.362+0000 7f9e5f200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f9e60072810 con 0x7f9e60162b80 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.362+0000 7f9e5f200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f9e600b49c0 con 0x7f9e60162f60 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.362+0000 7f9e5f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9e600b4ca0 con 0x7f9e600b4390 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.363+0000 7f9e5de00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9e60162b80 0x7f9e600b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.363+0000 7f9e5de00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9e60162b80 0x7f9e600b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47492/0 (socket says 172.21.15.49:47492) 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.363+0000 7f9e5de00640 1 -- 172.21.15.49:0/4244535764 learned_addr learned my addr 172.21.15.49:0/4244535764 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.363+0000 7f9e5e800640 1 --2- 172.21.15.49:0/4244535764 >> v2:172.21.15.107:3300/0 conn(0x7f9e60162f60 0x7f9e6006f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.363+0000 7f9e55400640 1 --2- 172.21.15.49:0/4244535764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e600b4390 0x7f9e60071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.363+0000 7f9e5de00640 1 -- 172.21.15.49:0/4244535764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e600b4390 msgr2=0x7f9e60071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.363+0000 7f9e5de00640 1 --2- 172.21.15.49:0/4244535764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e600b4390 0x7f9e60071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.363+0000 7f9e5de00640 1 -- 172.21.15.49:0/4244535764 >> v2:172.21.15.107:3300/0 conn(0x7f9e60162f60 msgr2=0x7f9e6006f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:31.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.363+0000 7f9e5de00640 1 --2- 172.21.15.49:0/4244535764 >> v2:172.21.15.107:3300/0 conn(0x7f9e60162f60 0x7f9e6006f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.363+0000 7f9e5de00640 1 -- 172.21.15.49:0/4244535764 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e600b4f80 con 0x7f9e60162b80 2024-04-15T15:19:31.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.365+0000 7f9e5de00640 1 --2- 172.21.15.49:0/4244535764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e60162b80 0x7f9e600b5780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e60070090 tx=0x7f9e5002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25492e3739ce2a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.365+0000 7f9e5d400640 1 -- 172.21.15.49:0/4244535764 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50035020 con 0x7f9e60162b80 2024-04-15T15:19:31.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.365+0000 7f9e5d400640 1 -- 172.21.15.49:0/4244535764 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e50031870 con 0x7f9e60162b80 2024-04-15T15:19:31.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.365+0000 7f9e5d400640 1 -- 172.21.15.49:0/4244535764 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50031c40 con 0x7f9e60162b80 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.366+0000 7f9e5f200640 1 -- 172.21.15.49:0/4244535764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e60162b80 msgr2=0x7f9e600b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.366+0000 7f9e5f200640 1 --2- 172.21.15.49:0/4244535764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e60162b80 0x7f9e600b5780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e60070090 tx=0x7f9e5002b910 comp rx=0 tx=0).stop 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.366+0000 7f9e5f200640 1 -- 172.21.15.49:0/4244535764 shutdown_connections 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.366+0000 7f9e5f200640 1 --2- 172.21.15.49:0/4244535764 >> v2:172.21.15.107:3300/0 conn(0x7f9e60162f60 0x7f9e6006f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.366+0000 7f9e5f200640 1 --2- 172.21.15.49:0/4244535764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e600b4390 0x7f9e60071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.366+0000 7f9e5f200640 1 --2- 172.21.15.49:0/4244535764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e60162b80 0x7f9e600b5780 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.366+0000 7f9e5f200640 1 -- 172.21.15.49:0/4244535764 >> 172.21.15.49:0/4244535764 conn(0x7f9e6006b930 msgr2=0x7f9e6006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.366+0000 7f9e5f200640 1 -- 172.21.15.49:0/4244535764 shutdown_connections 2024-04-15T15:19:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.366+0000 7f9e5f200640 1 -- 172.21.15.49:0/4244535764 wait complete. 2024-04-15T15:19:31.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.368+0000 7f9e5f200640 1 Processor -- start 2024-04-15T15:19:31.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.368+0000 7f9e5f200640 1 -- start start 2024-04-15T15:19:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e5f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e600b4390 0x7f9e600aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e5f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e60162b80 0x7f9e6014e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e5f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e60162f60 0x7f9e6006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e5f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e600725a0 con 0x7f9e600b4390 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e5f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60072420 con 0x7f9e60162b80 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e5f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6006f750 con 0x7f9e60162f60 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e5de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e600b4390 0x7f9e600aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e5de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e600b4390 0x7f9e600aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47502/0 (socket says 172.21.15.49:47502) 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e5de00640 1 -- 172.21.15.49:0/3131670362 learned_addr learned my addr 172.21.15.49:0/3131670362 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e5e800640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e60162f60 0x7f9e6006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.369+0000 7f9e55400640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e60162b80 0x7f9e6014e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.370+0000 7f9e5de00640 1 -- 172.21.15.49:0/3131670362 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e60162f60 msgr2=0x7f9e6006e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.370+0000 7f9e5de00640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e60162f60 0x7f9e6006e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.370+0000 7f9e5de00640 1 -- 172.21.15.49:0/3131670362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e60162b80 msgr2=0x7f9e6014e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.370+0000 7f9e5de00640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e60162b80 0x7f9e6014e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.370+0000 7f9e5de00640 1 -- 172.21.15.49:0/3131670362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e600af250 con 0x7f9e600b4390 2024-04-15T15:19:31.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.370+0000 7f9e5de00640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e600b4390 0x7f9e600aed10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e50000c00 tx=0x7f9e500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.371+0000 7f9e56a00640 1 -- 172.21.15.49:0/3131670362 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50035040 con 0x7f9e600b4390 2024-04-15T15:19:31.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.371+0000 7f9e56a00640 1 -- 172.21.15.49:0/3131670362 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e50030d00 con 0x7f9e600b4390 2024-04-15T15:19:31.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.371+0000 7f9e5f200640 1 -- 172.21.15.49:0/3131670362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e601527c0 con 0x7f9e600b4390 2024-04-15T15:19:31.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.371+0000 7f9e56a00640 1 -- 172.21.15.49:0/3131670362 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500456a0 con 0x7f9e600b4390 2024-04-15T15:19:31.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.371+0000 7f9e5f200640 1 -- 172.21.15.49:0/3131670362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e60152d80 con 0x7f9e600b4390 2024-04-15T15:19:31.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.372+0000 7f9e5f200640 1 -- 172.21.15.49:0/3131670362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e600755c0 con 0x7f9e600b4390 2024-04-15T15:19:31.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.373+0000 7f9e56a00640 1 -- 172.21.15.49:0/3131670362 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e50038020 con 0x7f9e600b4390 2024-04-15T15:19:31.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.374+0000 7f9e56a00640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9e3c06c8f0 0x7f9e3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:31.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.374+0000 7f9e56a00640 1 -- 172.21.15.49:0/3131670362 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9e500adf20 con 0x7f9e600b4390 2024-04-15T15:19:31.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.374+0000 7f9e55400640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9e3c06c8f0 0x7f9e3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:31.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.375+0000 7f9e55400640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9e3c06c8f0 0x7f9e3c06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e600b1310 tx=0x7f9e48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.379+0000 7f9e56a00640 1 -- 172.21.15.49:0/3131670362 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e5007c2c0 con 0x7f9e600b4390 2024-04-15T15:19:31.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.588+0000 7f9e5f200640 1 -- 172.21.15.49:0/3131670362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9e601620f0 con 0x7f9e600b4390 2024-04-15T15:19:31.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.595+0000 7f9e56a00640 1 -- 172.21.15.49:0/3131670362 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f9e5007bbc0 con 0x7f9e600b4390 2024-04-15T15:19:31.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 -- 172.21.15.49:0/3131670362 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9e3c06c8f0 msgr2=0x7f9e3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9e3c06c8f0 0x7f9e3c06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e600b1310 tx=0x7f9e48008040 comp rx=0 tx=0).stop 2024-04-15T15:19:31.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 -- 172.21.15.49:0/3131670362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e600b4390 msgr2=0x7f9e600aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:31.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e600b4390 0x7f9e600aed10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e50000c00 tx=0x7f9e500046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 -- 172.21.15.49:0/3131670362 shutdown_connections 2024-04-15T15:19:31.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f9e3c06c8f0 0x7f9e3c06edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e60162f60 0x7f9e6006e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e60162b80 0x7f9e6014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 --2- 172.21.15.49:0/3131670362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e600b4390 0x7f9e600aed10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:31.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 -- 172.21.15.49:0/3131670362 >> 172.21.15.49:0/3131670362 conn(0x7f9e6006b930 msgr2=0x7f9e600cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:31.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 -- 172.21.15.49:0/3131670362 shutdown_connections 2024-04-15T15:19:31.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:31.598+0000 7f9e5f200640 1 -- 172.21.15.49:0/3131670362 wait complete. 2024-04-15T15:19:31.620 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-15T15:19:31.620 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:19:31.760 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:19:31.880 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-15T15:19:31.880 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:19:31.880 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:31.943 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:19:31.943 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:32.000 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:19:32.001 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:32.053 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:32.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.281+0000 7f56f3200640 1 Processor -- start 2024-04-15T15:19:32.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.281+0000 7f56f3200640 1 -- start start 2024-04-15T15:19:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.282+0000 7f56f3200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f56f415edd0 0x7f56f40d3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.282+0000 7f56f3200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56f40b6210 0x7f56f40b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.282+0000 7f56f3200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56f40cff90 con 0x7f56f40cf260 2024-04-15T15:19:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.282+0000 7f56f3200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f56f40b6bb0 con 0x7f56f40b6210 2024-04-15T15:19:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.282+0000 7f56f3200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f56f40d0270 con 0x7f56f415edd0 2024-04-15T15:19:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.283+0000 7f56f1400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56f40b6210 0x7f56f40b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.283+0000 7f56f1400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56f40b6210 0x7f56f40b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47514/0 (socket says 172.21.15.49:47514) 2024-04-15T15:19:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.283+0000 7f56f1400640 1 -- 172.21.15.49:0/1195014713 learned_addr learned my addr 172.21.15.49:0/1195014713 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.283+0000 7f56f1e00640 1 --2- 172.21.15.49:0/1195014713 >> v2:172.21.15.107:3300/0 conn(0x7f56f415edd0 0x7f56f40d3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.283+0000 7f56f1400640 1 -- 172.21.15.49:0/1195014713 >> v2:172.21.15.107:3300/0 conn(0x7f56f415edd0 msgr2=0x7f56f40d3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.283+0000 7f56f1400640 1 --2- 172.21.15.49:0/1195014713 >> v2:172.21.15.107:3300/0 conn(0x7f56f415edd0 0x7f56f40d3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.283+0000 7f56f1400640 1 -- 172.21.15.49:0/1195014713 >> v1:172.21.15.107:6789/0 conn(0x7f56f40cf260 legacy=0x7f56f40cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.283+0000 7f56f1400640 1 -- 172.21.15.49:0/1195014713 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f40d2d70 con 0x7f56f40b6210 2024-04-15T15:19:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.285+0000 7f56f1400640 1 --2- 172.21.15.49:0/1195014713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 0x7f56f40b6670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f56f406dd10 tx=0x7f56dc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cf3d1fb82f665a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.285+0000 7f56f0a00640 1 -- 172.21.15.49:0/1195014713 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc02e070 con 0x7f56f40b6210 2024-04-15T15:19:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.285+0000 7f56f0a00640 1 -- 172.21.15.49:0/1195014713 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56dc037070 con 0x7f56f40b6210 2024-04-15T15:19:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.285+0000 7f56f0a00640 1 -- 172.21.15.49:0/1195014713 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc031a00 con 0x7f56f40b6210 2024-04-15T15:19:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.286+0000 7f56f3200640 1 -- 172.21.15.49:0/1195014713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 msgr2=0x7f56f40b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.286+0000 7f56f3200640 1 --2- 172.21.15.49:0/1195014713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 0x7f56f40b6670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f56f406dd10 tx=0x7f56dc02b880 comp rx=0 tx=0).stop 2024-04-15T15:19:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.286+0000 7f56f3200640 1 -- 172.21.15.49:0/1195014713 shutdown_connections 2024-04-15T15:19:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.286+0000 7f56f3200640 1 --2- 172.21.15.49:0/1195014713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 0x7f56f40b6670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.286+0000 7f56f3200640 1 --2- 172.21.15.49:0/1195014713 >> v2:172.21.15.107:3300/0 conn(0x7f56f415edd0 0x7f56f40d3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.286+0000 7f56f3200640 1 -- 172.21.15.49:0/1195014713 >> 172.21.15.49:0/1195014713 conn(0x7f56f40d1210 msgr2=0x7f56f40d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.286+0000 7f56f3200640 1 -- 172.21.15.49:0/1195014713 shutdown_connections 2024-04-15T15:19:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.286+0000 7f56f3200640 1 -- 172.21.15.49:0/1195014713 wait complete. 2024-04-15T15:19:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.288+0000 7f56f3200640 1 Processor -- start 2024-04-15T15:19:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.288+0000 7f56f3200640 1 -- start start 2024-04-15T15:19:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.289+0000 7f56f3200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 0x7f56f4151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.289+0000 7f56f3200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56f40cf260 0x7f56f40cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.289+0000 7f56f3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f415edd0 0x7f56f40cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.289+0000 7f56f3200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56f40d3d90 con 0x7f56f40b6210 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.289+0000 7f56f3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f56f40d3330 con 0x7f56f415edd0 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.289+0000 7f56f3200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f56f415f350 con 0x7f56f40cf260 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.289+0000 7f56f1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 0x7f56f4151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.289+0000 7f56f1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 0x7f56f4151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47530/0 (socket says 172.21.15.49:47530) 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.289+0000 7f56f1e00640 1 -- 172.21.15.49:0/3673352885 learned_addr learned my addr 172.21.15.49:0/3673352885 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56f1400640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56f40cf260 0x7f56f40cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56f1e00640 1 -- 172.21.15.49:0/3673352885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56f40cf260 msgr2=0x7f56f40cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56f2800640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f415edd0 0x7f56f40cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56f1e00640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56f40cf260 0x7f56f40cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56f1e00640 1 -- 172.21.15.49:0/3673352885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f415edd0 msgr2=0x7f56f40cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56f1e00640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f415edd0 0x7f56f40cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56f1e00640 1 -- 172.21.15.49:0/3673352885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f415f630 con 0x7f56f40b6210 2024-04-15T15:19:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56f1400640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56f40cf260 0x7f56f40cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56f1e00640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 0x7f56f4151040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f56e400d5f0 tx=0x7f56e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56ea000640 1 -- 172.21.15.49:0/3673352885 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e40125f0 con 0x7f56f40b6210 2024-04-15T15:19:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56ea000640 1 -- 172.21.15.49:0/3673352885 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e4019070 con 0x7f56f40b6210 2024-04-15T15:19:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.290+0000 7f56ea000640 1 -- 172.21.15.49:0/3673352885 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e40129d0 con 0x7f56f40b6210 2024-04-15T15:19:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.291+0000 7f56f3200640 1 -- 172.21.15.49:0/3673352885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f415f910 con 0x7f56f40b6210 2024-04-15T15:19:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.291+0000 7f56f3200640 1 -- 172.21.15.49:0/3673352885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f415fed0 con 0x7f56f40b6210 2024-04-15T15:19:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.292+0000 7f56f3200640 1 -- 172.21.15.49:0/3673352885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f40755c0 con 0x7f56f40b6210 2024-04-15T15:19:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.293+0000 7f56ea000640 1 -- 172.21.15.49:0/3673352885 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56e4012b70 con 0x7f56f40b6210 2024-04-15T15:19:32.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.294+0000 7f56ea000640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f56d006c8f0 0x7f56d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.294+0000 7f56ea000640 1 -- 172.21.15.49:0/3673352885 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f56e408c0f0 con 0x7f56f40b6210 2024-04-15T15:19:32.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.294+0000 7f56f1400640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f56d006c8f0 0x7f56d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.295+0000 7f56f1400640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f56d006c8f0 0x7f56d006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f56dc036000 tx=0x7f56dc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.299+0000 7f56ea000640 1 -- 172.21.15.49:0/3673352885 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56e405a400 con 0x7f56f40b6210 2024-04-15T15:19:32.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.503+0000 7f56f3200640 1 -- 172.21.15.49:0/3673352885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56f40d2760 con 0x7f56f40b6210 2024-04-15T15:19:32.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.506+0000 7f56ea000640 1 -- 172.21.15.49:0/3673352885 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16854 (secure 0 0 0) 0x7f56e4059d00 con 0x7f56f40b6210 2024-04-15T15:19:32.509 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:32.509 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:19:32.083681+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:32.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.509+0000 7f56f3200640 1 -- 172.21.15.49:0/3673352885 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f56d006c8f0 msgr2=0x7f56d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.509+0000 7f56f3200640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f56d006c8f0 0x7f56d006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f56dc036000 tx=0x7f56dc008030 comp rx=0 tx=0).stop 2024-04-15T15:19:32.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.509+0000 7f56f3200640 1 -- 172.21.15.49:0/3673352885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 msgr2=0x7f56f4151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.509+0000 7f56f3200640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 0x7f56f4151040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f56e400d5f0 tx=0x7f56e400dab0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.509+0000 7f56f3200640 1 -- 172.21.15.49:0/3673352885 shutdown_connections 2024-04-15T15:19:32.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.509+0000 7f56f3200640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f56d006c8f0 0x7f56d006edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.509+0000 7f56f3200640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56f40cf260 0x7f56f40cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.509+0000 7f56f3200640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f415edd0 0x7f56f40cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.509+0000 7f56f3200640 1 --2- 172.21.15.49:0/3673352885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f40b6210 0x7f56f4151040 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.509+0000 7f56f3200640 1 -- 172.21.15.49:0/3673352885 >> 172.21.15.49:0/3673352885 conn(0x7f56f40d1210 msgr2=0x7f56f40b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:32.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.510+0000 7f56f3200640 1 -- 172.21.15.49:0/3673352885 shutdown_connections 2024-04-15T15:19:32.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.510+0000 7f56f3200640 1 -- 172.21.15.49:0/3673352885 wait complete. 2024-04-15T15:19:32.534 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:32.534 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:32.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.769+0000 7f90bd200640 1 Processor -- start 2024-04-15T15:19:32.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.769+0000 7f90bd200640 1 -- start start 2024-04-15T15:19:32.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.771+0000 7f90bd200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90b815a440 0x7f90b815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.771+0000 7f90bd200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90b8072150 0x7f90b80725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.771+0000 7f90bd200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b815d520 con 0x7f90b806fc80 2024-04-15T15:19:32.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.771+0000 7f90bd200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f90b8072b00 con 0x7f90b8072150 2024-04-15T15:19:32.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.771+0000 7f90bd200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f90b806e280 con 0x7f90b815a440 2024-04-15T15:19:32.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.772+0000 7f90b6800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90b8072150 0x7f90b80725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.772+0000 7f90ad400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f90b806fc80 0x7f90b80cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45924/0 (socket says 172.21.15.49:45924) 2024-04-15T15:19:32.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.772+0000 7f90ad400640 1 -- 172.21.15.49:0/3258329660 learned_addr learned my addr 172.21.15.49:0/3258329660 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.772+0000 7f90b5e00640 1 --2- 172.21.15.49:0/3258329660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90b815a440 0x7f90b815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.772+0000 7f90b6800640 1 -- 172.21.15.49:0/3258329660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90b815a440 msgr2=0x7f90b815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.772+0000 7f90b6800640 1 --2- 172.21.15.49:0/3258329660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90b815a440 0x7f90b815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.772+0000 7f90b6800640 1 -- 172.21.15.49:0/3258329660 >> v1:172.21.15.49:6789/0 conn(0x7f90b806fc80 legacy=0x7f90b80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.772+0000 7f90b6800640 1 -- 172.21.15.49:0/3258329660 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b815d800 con 0x7f90b8072150 2024-04-15T15:19:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.772+0000 7f90b5400640 1 -- 172.21.15.49:0/3258329660 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3050964879 0 0) 0x7f90b815d520 con 0x7f90b806fc80 2024-04-15T15:19:32.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.774+0000 7f90b6800640 1 --2- 172.21.15.49:0/3258329660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b8072150 0x7f90b80725c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009d90 tx=0x7f90b002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5eba43d6b2c5e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.774+0000 7f90b5400640 1 -- 172.21.15.49:0/3258329660 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0039070 con 0x7f90b8072150 2024-04-15T15:19:32.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.774+0000 7f90b5400640 1 -- 172.21.15.49:0/3258329660 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b002cc50 con 0x7f90b8072150 2024-04-15T15:19:32.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.774+0000 7f90b5400640 1 -- 172.21.15.49:0/3258329660 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0034440 con 0x7f90b8072150 2024-04-15T15:19:32.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.774+0000 7f90bd200640 1 -- 172.21.15.49:0/3258329660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b8072150 msgr2=0x7f90b80725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.774+0000 7f90bd200640 1 --2- 172.21.15.49:0/3258329660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b8072150 0x7f90b80725c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009d90 tx=0x7f90b002bea0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.775+0000 7f90bd200640 1 -- 172.21.15.49:0/3258329660 shutdown_connections 2024-04-15T15:19:32.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.775+0000 7f90bd200640 1 --2- 172.21.15.49:0/3258329660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b8072150 0x7f90b80725c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.775+0000 7f90bd200640 1 --2- 172.21.15.49:0/3258329660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90b815a440 0x7f90b815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.775+0000 7f90bd200640 1 -- 172.21.15.49:0/3258329660 >> 172.21.15.49:0/3258329660 conn(0x7f90b80b2460 msgr2=0x7f90b80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:32.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.775+0000 7f90bd200640 1 -- 172.21.15.49:0/3258329660 shutdown_connections 2024-04-15T15:19:32.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.775+0000 7f90bd200640 1 -- 172.21.15.49:0/3258329660 wait complete. 2024-04-15T15:19:32.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.776+0000 7f90bd200640 1 Processor -- start 2024-04-15T15:19:32.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.777+0000 7f90bd200640 1 -- start start 2024-04-15T15:19:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.777+0000 7f90bd200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806fc80 0x7f90b8144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.777+0000 7f90bd200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b8072150 0x7f90b80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90bd200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90b815a440 0x7f90b81f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90bd200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90b815a820 con 0x7f90b806fc80 2024-04-15T15:19:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90bd200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90b80cd7f0 con 0x7f90b8072150 2024-04-15T15:19:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90bd200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f90b815ddc0 con 0x7f90b815a440 2024-04-15T15:19:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90b5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806fc80 0x7f90b8144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90b5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806fc80 0x7f90b8144cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47548/0 (socket says 172.21.15.49:47548) 2024-04-15T15:19:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90b5e00640 1 -- 172.21.15.49:0/60776236 learned_addr learned my addr 172.21.15.49:0/60776236 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90b6800640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90b815a440 0x7f90b81f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90ad400640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b8072150 0x7f90b80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90b5e00640 1 -- 172.21.15.49:0/60776236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90b815a440 msgr2=0x7f90b81f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90b5e00640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90b815a440 0x7f90b81f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90b5e00640 1 -- 172.21.15.49:0/60776236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b8072150 msgr2=0x7f90b80c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90b5e00640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b8072150 0x7f90b80c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90b5e00640 1 -- 172.21.15.49:0/60776236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b80c7230 con 0x7f90b806fc80 2024-04-15T15:19:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.778+0000 7f90ad400640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b8072150 0x7f90b80c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.779+0000 7f90b5e00640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806fc80 0x7f90b8144cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90a800e870 tx=0x7f90a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.779+0000 7f90aea00640 1 -- 172.21.15.49:0/60776236 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8017070 con 0x7f90b806fc80 2024-04-15T15:19:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.779+0000 7f90aea00640 1 -- 172.21.15.49:0/60776236 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a800c940 con 0x7f90b806fc80 2024-04-15T15:19:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.779+0000 7f90bd200640 1 -- 172.21.15.49:0/60776236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b80c7510 con 0x7f90b806fc80 2024-04-15T15:19:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.779+0000 7f90aea00640 1 -- 172.21.15.49:0/60776236 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a800cd30 con 0x7f90b806fc80 2024-04-15T15:19:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.779+0000 7f90bd200640 1 -- 172.21.15.49:0/60776236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b81451f0 con 0x7f90b806fc80 2024-04-15T15:19:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.780+0000 7f90bd200640 1 -- 172.21.15.49:0/60776236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b815d5c0 con 0x7f90b806fc80 2024-04-15T15:19:32.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.782+0000 7f90aea00640 1 -- 172.21.15.49:0/60776236 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90a801a070 con 0x7f90b806fc80 2024-04-15T15:19:32.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.783+0000 7f90aea00640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f909406c8f0 0x7f909406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:32.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.783+0000 7f90aea00640 1 -- 172.21.15.49:0/60776236 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f90a808bd90 con 0x7f90b806fc80 2024-04-15T15:19:32.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.783+0000 7f90ad400640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f909406c8f0 0x7f909406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:32.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.784+0000 7f90ad400640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f909406c8f0 0x7f909406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f90b80d0090 tx=0x7f90a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:32.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.788+0000 7f90aea00640 1 -- 172.21.15.49:0/60776236 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90b815d5c0 con 0x7f90b806fc80 2024-04-15T15:19:32.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.997+0000 7f90bd200640 1 -- 172.21.15.49:0/60776236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f90b8068c80 con 0x7f90b806fc80 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:32.998+0000 7f90aea00640 1 -- 172.21.15.49:0/60776236 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f90a805a1d0 con 0x7f90b806fc80 2024-04-15T15:19:33.000 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 -- 172.21.15.49:0/60776236 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f909406c8f0 msgr2=0x7f909406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f909406c8f0 0x7f909406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f90b80d0090 tx=0x7f90a0009040 comp rx=0 tx=0).stop 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 -- 172.21.15.49:0/60776236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806fc80 msgr2=0x7f90b8144cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806fc80 0x7f90b8144cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90a800e870 tx=0x7f90a800ed30 comp rx=0 tx=0).stop 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 -- 172.21.15.49:0/60776236 shutdown_connections 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f909406c8f0 0x7f909406edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90b815a440 0x7f90b81f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b8072150 0x7f90b80c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 --2- 172.21.15.49:0/60776236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806fc80 0x7f90b8144cb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 -- 172.21.15.49:0/60776236 >> 172.21.15.49:0/60776236 conn(0x7f90b80b2460 msgr2=0x7f90b80cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 -- 172.21.15.49:0/60776236 shutdown_connections 2024-04-15T15:19:33.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.001+0000 7f90bd200640 1 -- 172.21.15.49:0/60776236 wait complete. 2024-04-15T15:19:33.024 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:33.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.255+0000 7f12ee600640 1 Processor -- start 2024-04-15T15:19:33.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.255+0000 7f12ee600640 1 -- start start 2024-04-15T15:19:33.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.257+0000 7f12ee600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f12e81461c0 0x7f12e8071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.257+0000 7f12ee600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f12e80cebb0 0x7f12e806df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.257+0000 7f12ee600640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e80cd2d0 con 0x7f12e8072420 2024-04-15T15:19:33.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.257+0000 7f12ee600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f12e80cf1d0 con 0x7f12e81461c0 2024-04-15T15:19:33.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.257+0000 7f12ee600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f12e8072880 con 0x7f12e80cebb0 2024-04-15T15:19:33.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.257+0000 7f12e7e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f12e8072420 0x7f12e80ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45928/0 (socket says 172.21.15.49:45928) 2024-04-15T15:19:33.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.257+0000 7f12e7e00640 1 -- 172.21.15.49:0/2888208115 learned_addr learned my addr 172.21.15.49:0/2888208115 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:33.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.257+0000 7f12ed200640 1 --2- 172.21.15.49:0/2888208115 >> v2:172.21.15.49:3300/0 conn(0x7f12e81461c0 0x7f12e8071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.258+0000 7f12edc00640 1 --2- 172.21.15.49:0/2888208115 >> v2:172.21.15.107:3300/0 conn(0x7f12e80cebb0 0x7f12e806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.258+0000 7f12e7400640 1 -- 172.21.15.49:0/2888208115 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1730666356 0 0) 0x7f12e80cd2d0 con 0x7f12e8072420 2024-04-15T15:19:33.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.258+0000 7f12e7400640 1 -- 172.21.15.49:0/2888208115 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12d0003520 con 0x7f12e8072420 2024-04-15T15:19:33.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.258+0000 7f12ed200640 1 -- 172.21.15.49:0/2888208115 >> v2:172.21.15.107:3300/0 conn(0x7f12e80cebb0 msgr2=0x7f12e806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:33.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.258+0000 7f12ed200640 1 --2- 172.21.15.49:0/2888208115 >> v2:172.21.15.107:3300/0 conn(0x7f12e80cebb0 0x7f12e806df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.258+0000 7f12ed200640 1 -- 172.21.15.49:0/2888208115 >> v1:172.21.15.49:6789/0 conn(0x7f12e8072420 legacy=0x7f12e80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.258+0000 7f12ed200640 1 -- 172.21.15.49:0/2888208115 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e80cd5b0 con 0x7f12e81461c0 2024-04-15T15:19:33.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.259+0000 7f12ed200640 1 --2- 172.21.15.49:0/2888208115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 0x7f12e8071ee0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f12e8071d30 tx=0x7f12dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d3c41834925673 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.260+0000 7f12e7400640 1 -- 172.21.15.49:0/2888208115 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc035020 con 0x7f12e81461c0 2024-04-15T15:19:33.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.260+0000 7f12e7400640 1 -- 172.21.15.49:0/2888208115 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12dc031830 con 0x7f12e81461c0 2024-04-15T15:19:33.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.260+0000 7f12e7400640 1 -- 172.21.15.49:0/2888208115 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc031be0 con 0x7f12e81461c0 2024-04-15T15:19:33.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.260+0000 7f12ee600640 1 -- 172.21.15.49:0/2888208115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 msgr2=0x7f12e8071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.260+0000 7f12ee600640 1 --2- 172.21.15.49:0/2888208115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 0x7f12e8071ee0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f12e8071d30 tx=0x7f12dc02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.261+0000 7f12ee600640 1 -- 172.21.15.49:0/2888208115 shutdown_connections 2024-04-15T15:19:33.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.261+0000 7f12ee600640 1 --2- 172.21.15.49:0/2888208115 >> v2:172.21.15.107:3300/0 conn(0x7f12e80cebb0 0x7f12e806df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.261+0000 7f12ee600640 1 --2- 172.21.15.49:0/2888208115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 0x7f12e8071ee0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.261+0000 7f12ee600640 1 -- 172.21.15.49:0/2888208115 >> 172.21.15.49:0/2888208115 conn(0x7f12e806b930 msgr2=0x7f12e806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.261+0000 7f12ee600640 1 -- 172.21.15.49:0/2888208115 shutdown_connections 2024-04-15T15:19:33.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.261+0000 7f12ee600640 1 -- 172.21.15.49:0/2888208115 wait complete. 2024-04-15T15:19:33.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.263+0000 7f12ee600640 1 Processor -- start 2024-04-15T15:19:33.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.263+0000 7f12ee600640 1 -- start start 2024-04-15T15:19:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12ee600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e8072420 0x7f12e8159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12ee600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e80cebb0 0x7f12e81e5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12ee600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 0x7f12e8155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12ee600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80cdb70 con 0x7f12e81461c0 2024-04-15T15:19:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12ee600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80cd080 con 0x7f12e80cebb0 2024-04-15T15:19:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12ee600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f12e806e460 con 0x7f12e8072420 2024-04-15T15:19:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12edc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 0x7f12e8155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12edc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 0x7f12e8155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47562/0 (socket says 172.21.15.49:47562) 2024-04-15T15:19:33.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12edc00640 1 -- 172.21.15.49:0/501696666 learned_addr learned my addr 172.21.15.49:0/501696666 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:33.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12ed200640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e8072420 0x7f12e8159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12e7e00640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e80cebb0 0x7f12e81e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12edc00640 1 -- 172.21.15.49:0/501696666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e8072420 msgr2=0x7f12e8159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12edc00640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e8072420 0x7f12e8159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12edc00640 1 -- 172.21.15.49:0/501696666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e80cebb0 msgr2=0x7f12e81e5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.264+0000 7f12edc00640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e80cebb0 0x7f12e81e5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.265+0000 7f12edc00640 1 -- 172.21.15.49:0/501696666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e815a150 con 0x7f12e81461c0 2024-04-15T15:19:33.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.265+0000 7f12edc00640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 0x7f12e8155a40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f12e000efc0 tx=0x7f12e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.265+0000 7f12e4c00640 1 -- 172.21.15.49:0/501696666 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0018020 con 0x7f12e81461c0 2024-04-15T15:19:33.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.265+0000 7f12e4c00640 1 -- 172.21.15.49:0/501696666 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e0014480 con 0x7f12e81461c0 2024-04-15T15:19:33.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.265+0000 7f12ee600640 1 -- 172.21.15.49:0/501696666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e815a430 con 0x7f12e81461c0 2024-04-15T15:19:33.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.265+0000 7f12e4c00640 1 -- 172.21.15.49:0/501696666 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0014870 con 0x7f12e81461c0 2024-04-15T15:19:33.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.265+0000 7f12ee600640 1 -- 172.21.15.49:0/501696666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e81e2c60 con 0x7f12e81461c0 2024-04-15T15:19:33.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.267+0000 7f12ee600640 1 -- 172.21.15.49:0/501696666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e80cc8b0 con 0x7f12e81461c0 2024-04-15T15:19:33.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.267+0000 7f12e4c00640 1 -- 172.21.15.49:0/501696666 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12e0005d10 con 0x7f12e81461c0 2024-04-15T15:19:33.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.268+0000 7f12e4c00640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f12c806c8f0 0x7f12c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.269+0000 7f12e4c00640 1 -- 172.21.15.49:0/501696666 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f12e008c800 con 0x7f12e81461c0 2024-04-15T15:19:33.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.269+0000 7f12ed200640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f12c806c8f0 0x7f12c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.270+0000 7f12ed200640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f12c806c8f0 0x7f12c806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f12dc036000 tx=0x7f12dc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.273+0000 7f12e4c00640 1 -- 172.21.15.49:0/501696666 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12e80cc8b0 con 0x7f12e81461c0 2024-04-15T15:19:33.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.482+0000 7f12ee600640 1 -- 172.21.15.49:0/501696666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f12e80755c0 con 0x7f12e81461c0 2024-04-15T15:19:33.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.483+0000 7f12e4c00640 1 -- 172.21.15.49:0/501696666 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f12e0022070 con 0x7f12e81461c0 2024-04-15T15:19:33.484 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 -- 172.21.15.49:0/501696666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f12c806c8f0 msgr2=0x7f12c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f12c806c8f0 0x7f12c806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f12dc036000 tx=0x7f12dc008030 comp rx=0 tx=0).stop 2024-04-15T15:19:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 -- 172.21.15.49:0/501696666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 msgr2=0x7f12e8155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 0x7f12e8155a40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f12e000efc0 tx=0x7f12e000c490 comp rx=0 tx=0).stop 2024-04-15T15:19:33.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 -- 172.21.15.49:0/501696666 shutdown_connections 2024-04-15T15:19:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f12c806c8f0 0x7f12c806edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12e81461c0 0x7f12e8155a40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e80cebb0 0x7f12e81e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 --2- 172.21.15.49:0/501696666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12e8072420 0x7f12e8159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 -- 172.21.15.49:0/501696666 >> 172.21.15.49:0/501696666 conn(0x7f12e806b930 msgr2=0x7f12e80b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.485+0000 7f12ee600640 1 -- 172.21.15.49:0/501696666 shutdown_connections 2024-04-15T15:19:33.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.486+0000 7f12ee600640 1 -- 172.21.15.49:0/501696666 wait complete. 2024-04-15T15:19:33.508 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:33.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.744+0000 7fc978c00640 1 Processor -- start 2024-04-15T15:19:33.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.744+0000 7fc978c00640 1 -- start start 2024-04-15T15:19:33.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.746+0000 7fc978c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc974162f90 0x7fc9740ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.746+0000 7fc978c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc974072860 0x7fc974166e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.746+0000 7fc978c00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9740cec10 con 0x7fc9740afbc0 2024-04-15T15:19:33.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.746+0000 7fc978c00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc9740b46c0 con 0x7fc974162f90 2024-04-15T15:19:33.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.746+0000 7fc978c00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fc97406fe90 con 0x7fc974072860 2024-04-15T15:19:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.746+0000 7fc971400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc9740afbc0 0x7fc974072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45936/0 (socket says 172.21.15.49:45936) 2024-04-15T15:19:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.746+0000 7fc971400640 1 -- 172.21.15.49:0/1289583590 learned_addr learned my addr 172.21.15.49:0/1289583590 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.746+0000 7fc971e00640 1 --2- 172.21.15.49:0/1289583590 >> v2:172.21.15.49:3300/0 conn(0x7fc974162f90 0x7fc9740ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.747+0000 7fc972800640 1 --2- 172.21.15.49:0/1289583590 >> v2:172.21.15.107:3300/0 conn(0x7fc974072860 0x7fc974166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.747+0000 7fc970a00640 1 -- 172.21.15.49:0/1289583590 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1601336107 0 0) 0x7fc9740cec10 con 0x7fc9740afbc0 2024-04-15T15:19:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.747+0000 7fc970a00640 1 -- 172.21.15.49:0/1289583590 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc958003520 con 0x7fc9740afbc0 2024-04-15T15:19:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.747+0000 7fc971e00640 1 -- 172.21.15.49:0/1289583590 >> v2:172.21.15.107:3300/0 conn(0x7fc974072860 msgr2=0x7fc974166e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.747+0000 7fc971e00640 1 --2- 172.21.15.49:0/1289583590 >> v2:172.21.15.107:3300/0 conn(0x7fc974072860 0x7fc974166e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.747+0000 7fc971e00640 1 -- 172.21.15.49:0/1289583590 >> v1:172.21.15.49:6789/0 conn(0x7fc9740afbc0 legacy=0x7fc974072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.747+0000 7fc971e00640 1 -- 172.21.15.49:0/1289583590 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9740b3ab0 con 0x7fc974162f90 2024-04-15T15:19:33.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.748+0000 7fc971e00640 1 --2- 172.21.15.49:0/1289583590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 0x7fc9740ce6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc974071f10 tx=0x7fc96402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6239a1cfe2abf1d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.748+0000 7fc970a00640 1 -- 172.21.15.49:0/1289583590 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964035020 con 0x7fc974162f90 2024-04-15T15:19:33.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.749+0000 7fc970a00640 1 -- 172.21.15.49:0/1289583590 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc964031830 con 0x7fc974162f90 2024-04-15T15:19:33.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.749+0000 7fc970a00640 1 -- 172.21.15.49:0/1289583590 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964031c00 con 0x7fc974162f90 2024-04-15T15:19:33.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.749+0000 7fc978c00640 1 -- 172.21.15.49:0/1289583590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 msgr2=0x7fc9740ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.749+0000 7fc978c00640 1 --2- 172.21.15.49:0/1289583590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 0x7fc9740ce6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc974071f10 tx=0x7fc96402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.749+0000 7fc978c00640 1 -- 172.21.15.49:0/1289583590 shutdown_connections 2024-04-15T15:19:33.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.749+0000 7fc978c00640 1 --2- 172.21.15.49:0/1289583590 >> v2:172.21.15.107:3300/0 conn(0x7fc974072860 0x7fc974166e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.749+0000 7fc978c00640 1 --2- 172.21.15.49:0/1289583590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 0x7fc9740ce6d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.749+0000 7fc978c00640 1 -- 172.21.15.49:0/1289583590 >> 172.21.15.49:0/1289583590 conn(0x7fc9740b2460 msgr2=0x7fc9740b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.750+0000 7fc978c00640 1 -- 172.21.15.49:0/1289583590 shutdown_connections 2024-04-15T15:19:33.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.750+0000 7fc978c00640 1 -- 172.21.15.49:0/1289583590 wait complete. 2024-04-15T15:19:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.751+0000 7fc978c00640 1 Processor -- start 2024-04-15T15:19:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.752+0000 7fc978c00640 1 -- start start 2024-04-15T15:19:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.752+0000 7fc978c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc974072860 0x7fc9741f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.752+0000 7fc978c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9740afbc0 0x7fc97415fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.752+0000 7fc978c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 0x7fc9741fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc978c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc9740b4350 con 0x7fc974162f90 2024-04-15T15:19:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc978c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9740b4070 con 0x7fc974072860 2024-04-15T15:19:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc978c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc974163370 con 0x7fc9740afbc0 2024-04-15T15:19:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc972800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 0x7fc9741fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc972800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 0x7fc9741fcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47574/0 (socket says 172.21.15.49:47574) 2024-04-15T15:19:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc972800640 1 -- 172.21.15.49:0/1616113867 learned_addr learned my addr 172.21.15.49:0/1616113867 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc971e00640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc974072860 0x7fc9741f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc971400640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9740afbc0 0x7fc97415fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc972800640 1 -- 172.21.15.49:0/1616113867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9740afbc0 msgr2=0x7fc97415fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc972800640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9740afbc0 0x7fc97415fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc972800640 1 -- 172.21.15.49:0/1616113867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc974072860 msgr2=0x7fc9741f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc972800640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc974072860 0x7fc9741f8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc972800640 1 -- 172.21.15.49:0/1616113867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9741fd030 con 0x7fc974162f90 2024-04-15T15:19:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.753+0000 7fc971e00640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc974072860 0x7fc9741f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.754+0000 7fc972800640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 0x7fc9741fcaf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc96c007e40 tx=0x7fc96c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.754+0000 7fc96a000640 1 -- 172.21.15.49:0/1616113867 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c014620 con 0x7fc974162f90 2024-04-15T15:19:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.754+0000 7fc96a000640 1 -- 172.21.15.49:0/1616113867 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96c019070 con 0x7fc974162f90 2024-04-15T15:19:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.754+0000 7fc96a000640 1 -- 172.21.15.49:0/1616113867 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c014a20 con 0x7fc974162f90 2024-04-15T15:19:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.754+0000 7fc978c00640 1 -- 172.21.15.49:0/1616113867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9741f9100 con 0x7fc974162f90 2024-04-15T15:19:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.754+0000 7fc978c00640 1 -- 172.21.15.49:0/1616113867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9741f96c0 con 0x7fc974162f90 2024-04-15T15:19:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.756+0000 7fc978c00640 1 -- 172.21.15.49:0/1616113867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc974068c80 con 0x7fc974162f90 2024-04-15T15:19:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.757+0000 7fc96a000640 1 -- 172.21.15.49:0/1616113867 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc96c014bc0 con 0x7fc974162f90 2024-04-15T15:19:33.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.757+0000 7fc96a000640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc95406c940 0x7fc95406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:33.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.758+0000 7fc96a000640 1 -- 172.21.15.49:0/1616113867 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fc96c08c500 con 0x7fc974162f90 2024-04-15T15:19:33.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.758+0000 7fc971e00640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc95406c940 0x7fc95406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:33.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.758+0000 7fc971e00640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc95406c940 0x7fc95406ee00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc964036000 tx=0x7fc964008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:33.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.762+0000 7fc96a000640 1 -- 172.21.15.49:0/1616113867 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9741fd030 con 0x7fc974162f90 2024-04-15T15:19:33.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.969+0000 7fc978c00640 1 -- 172.21.15.49:0/1616113867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc97406f8d0 con 0x7fc974162f90 2024-04-15T15:19:33.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.970+0000 7fc96a000640 1 -- 172.21.15.49:0/1616113867 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fc96c05a890 con 0x7fc974162f90 2024-04-15T15:19:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 -- 172.21.15.49:0/1616113867 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc95406c940 msgr2=0x7fc95406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc95406c940 0x7fc95406ee00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc964036000 tx=0x7fc964008030 comp rx=0 tx=0).stop 2024-04-15T15:19:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 -- 172.21.15.49:0/1616113867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 msgr2=0x7fc9741fcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 0x7fc9741fcaf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc96c007e40 tx=0x7fc96c00e4b0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 -- 172.21.15.49:0/1616113867 shutdown_connections 2024-04-15T15:19:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc95406c940 0x7fc95406ee00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9740afbc0 0x7fc97415fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc974162f90 0x7fc9741fcaf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 --2- 172.21.15.49:0/1616113867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc974072860 0x7fc9741f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 -- 172.21.15.49:0/1616113867 >> 172.21.15.49:0/1616113867 conn(0x7fc9740b2460 msgr2=0x7fc9740cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:33.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 -- 172.21.15.49:0/1616113867 shutdown_connections 2024-04-15T15:19:33.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:33.973+0000 7fc978c00640 1 -- 172.21.15.49:0/1616113867 wait complete. 2024-04-15T15:19:33.992 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:19:34.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.208+0000 7f8f01000640 1 Processor -- start 2024-04-15T15:19:34.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.209+0000 7f8f01000640 1 -- start start 2024-04-15T15:19:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.210+0000 7f8f01000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8efc15df30 0x7f8efc166270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.210+0000 7f8f01000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8efc0cd360 con 0x7f8efc15b410 2024-04-15T15:19:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.211+0000 7f8f01000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8efc0cd640 con 0x7f8efc0cf530 2024-04-15T15:19:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.211+0000 7f8f01000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f8efc0cd920 con 0x7f8efc15df30 2024-04-15T15:19:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.211+0000 7f8efb400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8efc15b410 0x7f8efc15c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37634/0 (socket says 172.21.15.49:37634) 2024-04-15T15:19:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.211+0000 7f8efb400640 1 -- 172.21.15.49:0/1472770462 learned_addr learned my addr 172.21.15.49:0/1472770462 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.211+0000 7f8efbe00640 1 --2- 172.21.15.49:0/1472770462 >> v2:172.21.15.49:3300/0 conn(0x7f8efc15df30 0x7f8efc166270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.211+0000 7f8efa000640 1 -- 172.21.15.49:0/1472770462 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575825524 0 0) 0x7f8efc0cd360 con 0x7f8efc15b410 2024-04-15T15:19:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.211+0000 7f8efa000640 1 -- 172.21.15.49:0/1472770462 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ee0005730 con 0x7f8efc15b410 2024-04-15T15:19:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.211+0000 7f8efbe00640 1 -- 172.21.15.49:0/1472770462 >> v1:172.21.15.107:6789/0 conn(0x7f8efc0cf530 legacy=0x7f8efc0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.211+0000 7f8efbe00640 1 -- 172.21.15.49:0/1472770462 >> v1:172.21.15.49:6789/0 conn(0x7f8efc15b410 legacy=0x7f8efc15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.212+0000 7f8efbe00640 1 -- 172.21.15.49:0/1472770462 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc0cdc00 con 0x7f8efc15df30 2024-04-15T15:19:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.213+0000 7f8efbe00640 1 --2- 172.21.15.49:0/1472770462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 0x7f8efc166270 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40097a0 tx=0x7f8ef402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626853c5d507f4c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.213+0000 7f8efa000640 1 -- 172.21.15.49:0/1472770462 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef40315a0 con 0x7f8efc15df30 2024-04-15T15:19:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.213+0000 7f8efa000640 1 -- 172.21.15.49:0/1472770462 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef4037070 con 0x7f8efc15df30 2024-04-15T15:19:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.213+0000 7f8efa000640 1 -- 172.21.15.49:0/1472770462 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4031960 con 0x7f8efc15df30 2024-04-15T15:19:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.214+0000 7f8f01000640 1 -- 172.21.15.49:0/1472770462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 msgr2=0x7f8efc166270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.214+0000 7f8f01000640 1 --2- 172.21.15.49:0/1472770462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 0x7f8efc166270 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40097a0 tx=0x7f8ef402b840 comp rx=0 tx=0).stop 2024-04-15T15:19:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.214+0000 7f8f01000640 1 -- 172.21.15.49:0/1472770462 shutdown_connections 2024-04-15T15:19:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.214+0000 7f8f01000640 1 --2- 172.21.15.49:0/1472770462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 0x7f8efc166270 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.214+0000 7f8f01000640 1 -- 172.21.15.49:0/1472770462 >> 172.21.15.49:0/1472770462 conn(0x7f8efc0b23a0 msgr2=0x7f8efc0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.214+0000 7f8f01000640 1 -- 172.21.15.49:0/1472770462 shutdown_connections 2024-04-15T15:19:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.214+0000 7f8f01000640 1 -- 172.21.15.49:0/1472770462 wait complete. 2024-04-15T15:19:34.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.216+0000 7f8f01000640 1 Processor -- start 2024-04-15T15:19:34.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.217+0000 7f8f01000640 1 -- start start 2024-04-15T15:19:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.218+0000 7f8f01000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8efc0cf530 0x7f8efc14b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.218+0000 7f8f01000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8efc15b410 0x7f8efc155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.218+0000 7f8f01000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 0x7f8efc155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.218+0000 7f8f01000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc0b03a0 con 0x7f8efc15df30 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.218+0000 7f8f01000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc0cd030 con 0x7f8efc0cf530 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.218+0000 7f8f01000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f8efc0b0520 con 0x7f8efc15b410 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.218+0000 7f8efbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 0x7f8efc155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.218+0000 7f8efbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 0x7f8efc155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38212/0 (socket says 172.21.15.49:38212) 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.218+0000 7f8efbe00640 1 -- 172.21.15.49:0/1510180553 learned_addr learned my addr 172.21.15.49:0/1510180553 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.219+0000 7f8efb400640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8efc0cf530 0x7f8efc14b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.219+0000 7f8efaa00640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8efc15b410 0x7f8efc155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.219+0000 7f8efbe00640 1 -- 172.21.15.49:0/1510180553 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8efc15b410 msgr2=0x7f8efc155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.219+0000 7f8efbe00640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8efc15b410 0x7f8efc155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.219+0000 7f8efbe00640 1 -- 172.21.15.49:0/1510180553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8efc0cf530 msgr2=0x7f8efc14b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.219+0000 7f8efbe00640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8efc0cf530 0x7f8efc14b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.219+0000 7f8efbe00640 1 -- 172.21.15.49:0/1510180553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc1592f0 con 0x7f8efc15df30 2024-04-15T15:19:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.219+0000 7f8efbe00640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 0x7f8efc155e90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4034040 tx=0x7f8ef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.220+0000 7f8ef3400640 1 -- 172.21.15.49:0/1510180553 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4049070 con 0x7f8efc15df30 2024-04-15T15:19:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.220+0000 7f8ef3400640 1 -- 172.21.15.49:0/1510180553 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef4032430 con 0x7f8efc15df30 2024-04-15T15:19:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.220+0000 7f8ef3400640 1 -- 172.21.15.49:0/1510180553 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4032800 con 0x7f8efc15df30 2024-04-15T15:19:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.220+0000 7f8f01000640 1 -- 172.21.15.49:0/1510180553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8efc1595d0 con 0x7f8efc15df30 2024-04-15T15:19:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.220+0000 7f8f01000640 1 -- 172.21.15.49:0/1510180553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8efc159b90 con 0x7f8efc15df30 2024-04-15T15:19:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.221+0000 7f8f01000640 1 -- 172.21.15.49:0/1510180553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8efc0755c0 con 0x7f8efc15df30 2024-04-15T15:19:34.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.222+0000 7f8ef3400640 1 -- 172.21.15.49:0/1510180553 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ef40329a0 con 0x7f8efc15df30 2024-04-15T15:19:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.224+0000 7f8ef3400640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ed806c8f0 0x7f8ed806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.224+0000 7f8ef3400640 1 -- 172.21.15.49:0/1510180553 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f8ef402f020 con 0x7f8efc15df30 2024-04-15T15:19:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.224+0000 7f8efb400640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ed806c8f0 0x7f8ed806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:34.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.225+0000 7f8efb400640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ed806c8f0 0x7f8ed806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8eec005c00 tx=0x7f8eec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:34.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.228+0000 7f8ef3400640 1 -- 172.21.15.49:0/1510180553 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ef407b680 con 0x7f8efc15df30 2024-04-15T15:19:34.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.436+0000 7f8f01000640 1 -- 172.21.15.49:0/1510180553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8efc15fe30 con 0x7f8efc15df30 2024-04-15T15:19:34.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.436+0000 7f8ef3400640 1 -- 172.21.15.49:0/1510180553 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f8ef407af80 con 0x7f8efc15df30 2024-04-15T15:19:34.438 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2024-04-15T15:19:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 -- 172.21.15.49:0/1510180553 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ed806c8f0 msgr2=0x7f8ed806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ed806c8f0 0x7f8ed806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8eec005c00 tx=0x7f8eec009040 comp rx=0 tx=0).stop 2024-04-15T15:19:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 -- 172.21.15.49:0/1510180553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 msgr2=0x7f8efc155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 0x7f8efc155e90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4034040 tx=0x7f8ef40046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 -- 172.21.15.49:0/1510180553 shutdown_connections 2024-04-15T15:19:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ed806c8f0 0x7f8ed806edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8efc15b410 0x7f8efc155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc15df30 0x7f8efc155e90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 --2- 172.21.15.49:0/1510180553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8efc0cf530 0x7f8efc14b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:34.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 -- 172.21.15.49:0/1510180553 >> 172.21.15.49:0/1510180553 conn(0x7f8efc0b23a0 msgr2=0x7f8efc0cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:34.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 -- 172.21.15.49:0/1510180553 shutdown_connections 2024-04-15T15:19:34.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:34.439+0000 7f8f01000640 1 -- 172.21.15.49:0/1510180553 wait complete. 2024-04-15T15:19:34.458 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-15T15:19:34.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:34.459 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-15T15:19:34.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:19:34.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:34.564 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-15T15:19:34.626 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:19:34.626 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:34.684 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:19:34.684 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:34.735 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:19:34.735 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:34.786 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:35.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.022+0000 7f113fe00640 1 Processor -- start 2024-04-15T15:19:35.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.023+0000 7f113fe00640 1 -- start start 2024-04-15T15:19:35.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.024+0000 7f113fe00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1140162020 0x7f1140162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.024+0000 7f113fe00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11400d0090 con 0x7f11400cf110 2024-04-15T15:19:35.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.024+0000 7f113fe00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11400cfad0 con 0x7f11400d1c90 2024-04-15T15:19:35.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.024+0000 7f113fe00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f1140163440 con 0x7f1140162020 2024-04-15T15:19:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.025+0000 7f113e000640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f11400cf110 0x7f11400d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37642/0 (socket says 172.21.15.49:37642) 2024-04-15T15:19:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.025+0000 7f113e000640 1 -- 172.21.15.49:0/1327598754 learned_addr learned my addr 172.21.15.49:0/1327598754 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:35.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.025+0000 7f113ea00640 1 --2- 172.21.15.49:0/1327598754 >> v2:172.21.15.107:3300/0 conn(0x7f1140162020 0x7f1140162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.025+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719381761 0 0) 0x7f1128001110 con 0x7f11400cf110 2024-04-15T15:19:35.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1124003520 con 0x7f11400cf110 2024-04-15T15:19:35.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2146002048 0 0) 0x7f1124003520 con 0x7f11400cf110 2024-04-15T15:19:35.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 >> v2:172.21.15.107:3300/0 conn(0x7f1140162020 msgr2=0x7f1140162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 --2- 172.21.15.49:0/1327598754 >> v2:172.21.15.107:3300/0 conn(0x7f1140162020 0x7f1140162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 >> v1:172.21.15.107:6789/0 conn(0x7f11400d1c90 legacy=0x7f11400d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11400cfdb0 con 0x7f11400cf110 2024-04-15T15:19:35.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f1128001840 con 0x7f11400cf110 2024-04-15T15:19:35.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 >> v1:172.21.15.49:6789/0 conn(0x7f11400cf110 legacy=0x7f11400d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1124003a00 0x7f1124003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1124004800 0x7f1124004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11240051c0 0x7f112400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1140163440 con 0x7f1124003a00 2024-04-15T15:19:35.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f112400c0c0 con 0x7f11240051c0 2024-04-15T15:19:35.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f112400c2c0 con 0x7f1124004800 2024-04-15T15:19:35.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.026+0000 7f113ea00640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1124003a00 0x7f1124003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.027+0000 7f113e000640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1124004800 0x7f1124004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.027+0000 7f113f400640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11240051c0 0x7f112400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.027+0000 7f113ea00640 1 -- 172.21.15.49:0/1327598754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1124004800 msgr2=0x7f1124004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.027+0000 7f113ea00640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1124004800 0x7f1124004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.027+0000 7f113ea00640 1 -- 172.21.15.49:0/1327598754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11240051c0 msgr2=0x7f112400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.027+0000 7f113ea00640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11240051c0 0x7f112400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.027+0000 7f113ea00640 1 -- 172.21.15.49:0/1327598754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1124003520 con 0x7f1124003a00 2024-04-15T15:19:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.027+0000 7f113f400640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11240051c0 0x7f112400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.027+0000 7f113f400640 1 -- 172.21.15.49:0/1327598754 reap_dead start 2024-04-15T15:19:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.028+0000 7f113ea00640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1124003a00 0x7f1124003e20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1130009ac0 tx=0x7f1124005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8fe080b77956a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.028+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130039070 con 0x7f1124003a00 2024-04-15T15:19:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.028+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113002c6b0 con 0x7f1124003a00 2024-04-15T15:19:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.028+0000 7f113d600640 1 -- 172.21.15.49:0/1327598754 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130009690 con 0x7f1124003a00 2024-04-15T15:19:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.029+0000 7f113fe00640 1 -- 172.21.15.49:0/1327598754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1124003a00 msgr2=0x7f1124003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.029+0000 7f113fe00640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1124003a00 0x7f1124003e20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1130009ac0 tx=0x7f1124005830 comp rx=0 tx=0).stop 2024-04-15T15:19:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.029+0000 7f113fe00640 1 -- 172.21.15.49:0/1327598754 shutdown_connections 2024-04-15T15:19:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.029+0000 7f113fe00640 1 --2- 172.21.15.49:0/1327598754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1124003a00 0x7f1124003e20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.029+0000 7f113fe00640 1 -- 172.21.15.49:0/1327598754 >> 172.21.15.49:0/1327598754 conn(0x7f114006cf00 msgr2=0x7f114006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.029+0000 7f113fe00640 1 -- 172.21.15.49:0/1327598754 shutdown_connections 2024-04-15T15:19:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.029+0000 7f113fe00640 1 -- 172.21.15.49:0/1327598754 wait complete. 2024-04-15T15:19:35.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.031+0000 7f113fe00640 1 Processor -- start 2024-04-15T15:19:35.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.031+0000 7f113fe00640 1 -- start start 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.032+0000 7f113fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1124003a00 0x7f11401f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.032+0000 7f113fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11401ea870 0x7f11401f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.032+0000 7f113fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11401f8700 0x7f11401f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.032+0000 7f113fe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11400cf880 con 0x7f11401f8700 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.032+0000 7f113fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11400b7b30 con 0x7f11401ea870 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.032+0000 7f113fe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f11400d2e70 con 0x7f1124003a00 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.032+0000 7f113f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11401f8700 0x7f11401f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.033+0000 7f113f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11401f8700 0x7f11401f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38242/0 (socket says 172.21.15.49:38242) 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.033+0000 7f113f400640 1 -- 172.21.15.49:0/1563932302 learned_addr learned my addr 172.21.15.49:0/1563932302 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.033+0000 7f113e000640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11401ea870 0x7f11401f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.033+0000 7f113f400640 1 -- 172.21.15.49:0/1563932302 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1124003a00 msgr2=0x7f11401f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.033+0000 7f113f400640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1124003a00 0x7f11401f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.033+0000 7f113f400640 1 -- 172.21.15.49:0/1563932302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11401ea870 msgr2=0x7f11401f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.033+0000 7f113f400640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11401ea870 0x7f11401f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.033+0000 7f113f400640 1 -- 172.21.15.49:0/1563932302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11401e8f30 con 0x7f11401f8700 2024-04-15T15:19:35.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.033+0000 7f113f400640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11401f8700 0x7f11401f4bd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f114008dc30 tx=0x7f113800d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.034+0000 7f1136a00640 1 -- 172.21.15.49:0/1563932302 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138018070 con 0x7f11401f8700 2024-04-15T15:19:35.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.034+0000 7f1136a00640 1 -- 172.21.15.49:0/1563932302 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1138006b50 con 0x7f11401f8700 2024-04-15T15:19:35.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.034+0000 7f1136a00640 1 -- 172.21.15.49:0/1563932302 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138003e60 con 0x7f11401f8700 2024-04-15T15:19:35.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.034+0000 7f113fe00640 1 -- 172.21.15.49:0/1563932302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11401e9210 con 0x7f11401f8700 2024-04-15T15:19:35.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.034+0000 7f113fe00640 1 -- 172.21.15.49:0/1563932302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11401e97d0 con 0x7f11401f8700 2024-04-15T15:19:35.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.035+0000 7f113fe00640 1 -- 172.21.15.49:0/1563932302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11400d45c0 con 0x7f11401f8700 2024-04-15T15:19:35.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.036+0000 7f1136a00640 1 -- 172.21.15.49:0/1563932302 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f113801b070 con 0x7f11401f8700 2024-04-15T15:19:35.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.037+0000 7f1136a00640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f111c06c8f0 0x7f111c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.037+0000 7f1136a00640 1 -- 172.21.15.49:0/1563932302 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f113808adb0 con 0x7f11401f8700 2024-04-15T15:19:35.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.038+0000 7f113ea00640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f111c06c8f0 0x7f111c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.038+0000 7f113ea00640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f111c06c8f0 0x7f111c06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1130036000 tx=0x7f113002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.042+0000 7f1136a00640 1 -- 172.21.15.49:0/1563932302 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f113808f070 con 0x7f11401f8700 2024-04-15T15:19:35.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.238+0000 7f113fe00640 1 -- 172.21.15.49:0/1563932302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11400755c0 con 0x7f11401f8700 2024-04-15T15:19:35.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.241+0000 7f1136a00640 1 -- 172.21.15.49:0/1563932302 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16854 (secure 0 0 0) 0x7f11380590c0 con 0x7f11401f8700 2024-04-15T15:19:35.243 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:35.243 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:19:32.083681+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:35.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.243+0000 7f113fe00640 1 -- 172.21.15.49:0/1563932302 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f111c06c8f0 msgr2=0x7f111c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.243+0000 7f113fe00640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f111c06c8f0 0x7f111c06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1130036000 tx=0x7f113002e000 comp rx=0 tx=0).stop 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.243+0000 7f113fe00640 1 -- 172.21.15.49:0/1563932302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11401f8700 msgr2=0x7f11401f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.243+0000 7f113fe00640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11401f8700 0x7f11401f4bd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f114008dc30 tx=0x7f113800d900 comp rx=0 tx=0).stop 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.244+0000 7f113fe00640 1 -- 172.21.15.49:0/1563932302 shutdown_connections 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.244+0000 7f113fe00640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f111c06c8f0 0x7f111c06edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.244+0000 7f113fe00640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11401f8700 0x7f11401f4bd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.244+0000 7f113fe00640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11401ea870 0x7f11401f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.244+0000 7f113fe00640 1 --2- 172.21.15.49:0/1563932302 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1124003a00 0x7f11401f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.244+0000 7f113fe00640 1 -- 172.21.15.49:0/1563932302 >> 172.21.15.49:0/1563932302 conn(0x7f114006cf00 msgr2=0x7f11400ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.244+0000 7f113fe00640 1 -- 172.21.15.49:0/1563932302 shutdown_connections 2024-04-15T15:19:35.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.244+0000 7f113fe00640 1 -- 172.21.15.49:0/1563932302 wait complete. 2024-04-15T15:19:35.262 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:35.263 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.480+0000 7f2137e00640 1 Processor -- start 2024-04-15T15:19:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.481+0000 7f2137e00640 1 -- start start 2024-04-15T15:19:35.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.482+0000 7f2137e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f213815d8f0 0x7f21380ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.482+0000 7f2137e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f21380ce850 0x7f21380b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.483+0000 7f2137e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21380cecd0 con 0x7f213815d510 2024-04-15T15:19:35.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.483+0000 7f2137e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f21380cefb0 con 0x7f21380ce850 2024-04-15T15:19:35.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.483+0000 7f2137e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f213806dfa0 con 0x7f213815d8f0 2024-04-15T15:19:35.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.483+0000 7f212fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f213815d8f0 0x7f21380ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.483+0000 7f212fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f213815d8f0 0x7f21380ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:34640/0 (socket says 172.21.15.49:34640) 2024-04-15T15:19:35.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.483+0000 7f212fe00640 1 -- 172.21.15.49:0/2142102560 learned_addr learned my addr 172.21.15.49:0/2142102560 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:35.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.483+0000 7f2137400640 1 --2- 172.21.15.49:0/2142102560 >> v2:172.21.15.107:3300/0 conn(0x7f21380ce850 0x7f21380b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.484+0000 7f212fe00640 1 -- 172.21.15.49:0/2142102560 >> v2:172.21.15.107:3300/0 conn(0x7f21380ce850 msgr2=0x7f21380b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.484+0000 7f212fe00640 1 --2- 172.21.15.49:0/2142102560 >> v2:172.21.15.107:3300/0 conn(0x7f21380ce850 0x7f21380b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.484+0000 7f212fe00640 1 -- 172.21.15.49:0/2142102560 >> v1:172.21.15.107:6789/0 conn(0x7f213815d510 legacy=0x7f213806ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.484+0000 7f212fe00640 1 -- 172.21.15.49:0/2142102560 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213806e280 con 0x7f213815d8f0 2024-04-15T15:19:35.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.484+0000 7f2136000640 1 -- 172.21.15.49:0/2142102560 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 298612286 0 0) 0x7f21380cecd0 con 0x7f213815d510 2024-04-15T15:19:35.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.486+0000 7f212fe00640 1 --2- 172.21.15.49:0/2142102560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f213815d8f0 0x7f21380ce310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2120009870 tx=0x7f212002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bbb401495ec05a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.486+0000 7f2136000640 1 -- 172.21.15.49:0/2142102560 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120034070 con 0x7f213815d8f0 2024-04-15T15:19:35.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.486+0000 7f2136000640 1 -- 172.21.15.49:0/2142102560 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2120030840 con 0x7f213815d8f0 2024-04-15T15:19:35.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.486+0000 7f2136000640 1 -- 172.21.15.49:0/2142102560 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120030c10 con 0x7f213815d8f0 2024-04-15T15:19:35.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.487+0000 7f2137e00640 1 -- 172.21.15.49:0/2142102560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f213815d8f0 msgr2=0x7f21380ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.487+0000 7f2137e00640 1 --2- 172.21.15.49:0/2142102560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f213815d8f0 0x7f21380ce310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2120009870 tx=0x7f212002b980 comp rx=0 tx=0).stop 2024-04-15T15:19:35.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.487+0000 7f2137e00640 1 -- 172.21.15.49:0/2142102560 shutdown_connections 2024-04-15T15:19:35.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.487+0000 7f2137e00640 1 --2- 172.21.15.49:0/2142102560 >> v2:172.21.15.107:3300/0 conn(0x7f21380ce850 0x7f21380b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.487+0000 7f2137e00640 1 --2- 172.21.15.49:0/2142102560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f213815d8f0 0x7f21380ce310 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.487+0000 7f2137e00640 1 -- 172.21.15.49:0/2142102560 >> 172.21.15.49:0/2142102560 conn(0x7f21380b23a0 msgr2=0x7f21380b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.487+0000 7f2137e00640 1 -- 172.21.15.49:0/2142102560 shutdown_connections 2024-04-15T15:19:35.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.487+0000 7f2137e00640 1 -- 172.21.15.49:0/2142102560 wait complete. 2024-04-15T15:19:35.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.489+0000 7f2137e00640 1 Processor -- start 2024-04-15T15:19:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.489+0000 7f2137e00640 1 -- start start 2024-04-15T15:19:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.490+0000 7f2137e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21380ce850 0x7f2138152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.490+0000 7f2137e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f213815d510 0x7f2138157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.490+0000 7f2137e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2138154470 0x7f2138143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.490+0000 7f2137e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f21380b6070 con 0x7f213815d510 2024-04-15T15:19:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.490+0000 7f2137e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f21380b5ef0 con 0x7f21380ce850 2024-04-15T15:19:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.490+0000 7f2137e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f21380b61f0 con 0x7f2138154470 2024-04-15T15:19:35.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.491+0000 7f212fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f213815d510 0x7f2138157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.491+0000 7f212fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f213815d510 0x7f2138157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38252/0 (socket says 172.21.15.49:38252) 2024-04-15T15:19:35.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.491+0000 7f212fe00640 1 -- 172.21.15.49:0/2758004352 learned_addr learned my addr 172.21.15.49:0/2758004352 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.491+0000 7f2136a00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21380ce850 0x7f2138152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.491+0000 7f212fe00640 1 -- 172.21.15.49:0/2758004352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2138154470 msgr2=0x7f2138143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.491+0000 7f212fe00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2138154470 0x7f2138143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.491+0000 7f212fe00640 1 -- 172.21.15.49:0/2758004352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21380ce850 msgr2=0x7f2138152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.491+0000 7f212fe00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21380ce850 0x7f2138152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.491+0000 7f212fe00640 1 -- 172.21.15.49:0/2758004352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2138158100 con 0x7f213815d510 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.491+0000 7f2136a00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21380ce850 0x7f2138152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.492+0000 7f212fe00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f213815d510 0x7f2138157bc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f21200090b0 tx=0x7f21200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.492+0000 7f212ea00640 1 -- 172.21.15.49:0/2758004352 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120034040 con 0x7f213815d510 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.492+0000 7f212ea00640 1 -- 172.21.15.49:0/2758004352 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f212003ad20 con 0x7f213815d510 2024-04-15T15:19:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.492+0000 7f212ea00640 1 -- 172.21.15.49:0/2758004352 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120044610 con 0x7f213815d510 2024-04-15T15:19:35.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.492+0000 7f2137e00640 1 -- 172.21.15.49:0/2758004352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21381533d0 con 0x7f213815d510 2024-04-15T15:19:35.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.492+0000 7f2137e00640 1 -- 172.21.15.49:0/2758004352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2138153990 con 0x7f213815d510 2024-04-15T15:19:35.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.493+0000 7f2137e00640 1 -- 172.21.15.49:0/2758004352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21380755c0 con 0x7f213815d510 2024-04-15T15:19:35.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.495+0000 7f212ea00640 1 -- 172.21.15.49:0/2758004352 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f212004b060 con 0x7f213815d510 2024-04-15T15:19:35.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.495+0000 7f212ea00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f211406c8f0 0x7f211406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.496+0000 7f212ea00640 1 -- 172.21.15.49:0/2758004352 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f21200adb10 con 0x7f213815d510 2024-04-15T15:19:35.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.496+0000 7f2136a00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f211406c8f0 0x7f211406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.497+0000 7f2136a00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f211406c8f0 0x7f211406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2128000f60 tx=0x7f2128009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.501+0000 7f212ea00640 1 -- 172.21.15.49:0/2758004352 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f212003f070 con 0x7f213815d510 2024-04-15T15:19:35.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.703+0000 7f2137e00640 1 -- 172.21.15.49:0/2758004352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f21380b13c0 con 0x7f213815d510 2024-04-15T15:19:35.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.704+0000 7f212ea00640 1 -- 172.21.15.49:0/2758004352 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f212007beb0 con 0x7f213815d510 2024-04-15T15:19:35.706 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 -- 172.21.15.49:0/2758004352 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f211406c8f0 msgr2=0x7f211406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f211406c8f0 0x7f211406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2128000f60 tx=0x7f2128009040 comp rx=0 tx=0).stop 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 -- 172.21.15.49:0/2758004352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f213815d510 msgr2=0x7f2138157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f213815d510 0x7f2138157bc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f21200090b0 tx=0x7f21200046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 -- 172.21.15.49:0/2758004352 shutdown_connections 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f211406c8f0 0x7f211406edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2138154470 0x7f2138143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f213815d510 0x7f2138157bc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 --2- 172.21.15.49:0/2758004352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21380ce850 0x7f2138152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 -- 172.21.15.49:0/2758004352 >> 172.21.15.49:0/2758004352 conn(0x7f21380b23a0 msgr2=0x7f213815b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 -- 172.21.15.49:0/2758004352 shutdown_connections 2024-04-15T15:19:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.707+0000 7f2137e00640 1 -- 172.21.15.49:0/2758004352 wait complete. 2024-04-15T15:19:35.727 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:35.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.951+0000 7fb08e000640 1 Processor -- start 2024-04-15T15:19:35.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.951+0000 7fb08e000640 1 -- start start 2024-04-15T15:19:35.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.952+0000 7fb08e000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb08815c680 0x7fb088162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.952+0000 7fb08e000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0880d2b50 0x7fb0880d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.952+0000 7fb08e000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0880b70d0 con 0x7fb0880b6c70 2024-04-15T15:19:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.952+0000 7fb08e000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fb088162dd0 con 0x7fb08815c680 2024-04-15T15:19:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.952+0000 7fb08e000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb0881630b0 con 0x7fb0880d2b50 2024-04-15T15:19:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.953+0000 7fb087e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb0880b6c70 0x7fb08815f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37650/0 (socket says 172.21.15.49:37650) 2024-04-15T15:19:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.953+0000 7fb087e00640 1 -- 172.21.15.49:0/2098720628 learned_addr learned my addr 172.21.15.49:0/2098720628 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.953+0000 7fb08cc00640 1 --2- 172.21.15.49:0/2098720628 >> v2:172.21.15.107:3300/0 conn(0x7fb08815c680 0x7fb088162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.953+0000 7fb08d600640 1 --2- 172.21.15.49:0/2098720628 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0880d2b50 0x7fb0880d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.953+0000 7fb087400640 1 -- 172.21.15.49:0/2098720628 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 790406586 0 0) 0x7fb0880b70d0 con 0x7fb0880b6c70 2024-04-15T15:19:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.953+0000 7fb087400640 1 -- 172.21.15.49:0/2098720628 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb06c003520 con 0x7fb0880b6c70 2024-04-15T15:19:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.954+0000 7fb08cc00640 1 -- 172.21.15.49:0/2098720628 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0880d2b50 msgr2=0x7fb0880d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.954+0000 7fb08cc00640 1 --2- 172.21.15.49:0/2098720628 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0880d2b50 0x7fb0880d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.954+0000 7fb08cc00640 1 -- 172.21.15.49:0/2098720628 >> v1:172.21.15.49:6789/0 conn(0x7fb0880b6c70 legacy=0x7fb08815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.954+0000 7fb08cc00640 1 -- 172.21.15.49:0/2098720628 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb088163390 con 0x7fb08815c680 2024-04-15T15:19:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.955+0000 7fb08cc00640 1 --2- 172.21.15.49:0/2098720628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08815c680 0x7fb088162730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb0780097a0 tx=0x7fb07802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9bfe5b2640588d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.955+0000 7fb087400640 1 -- 172.21.15.49:0/2098720628 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078035020 con 0x7fb08815c680 2024-04-15T15:19:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.955+0000 7fb087400640 1 -- 172.21.15.49:0/2098720628 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb078031810 con 0x7fb08815c680 2024-04-15T15:19:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.955+0000 7fb087400640 1 -- 172.21.15.49:0/2098720628 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078031be0 con 0x7fb08815c680 2024-04-15T15:19:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.956+0000 7fb08e000640 1 -- 172.21.15.49:0/2098720628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08815c680 msgr2=0x7fb088162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.956+0000 7fb08e000640 1 --2- 172.21.15.49:0/2098720628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08815c680 0x7fb088162730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb0780097a0 tx=0x7fb07802b8b0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.956+0000 7fb08e000640 1 -- 172.21.15.49:0/2098720628 shutdown_connections 2024-04-15T15:19:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.956+0000 7fb08e000640 1 --2- 172.21.15.49:0/2098720628 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0880d2b50 0x7fb0880d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.956+0000 7fb08e000640 1 --2- 172.21.15.49:0/2098720628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08815c680 0x7fb088162730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.956+0000 7fb08e000640 1 -- 172.21.15.49:0/2098720628 >> 172.21.15.49:0/2098720628 conn(0x7fb088073480 msgr2=0x7fb088073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.956+0000 7fb08e000640 1 -- 172.21.15.49:0/2098720628 shutdown_connections 2024-04-15T15:19:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.956+0000 7fb08e000640 1 -- 172.21.15.49:0/2098720628 wait complete. 2024-04-15T15:19:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.958+0000 7fb08e000640 1 Processor -- start 2024-04-15T15:19:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.958+0000 7fb08e000640 1 -- start start 2024-04-15T15:19:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.959+0000 7fb08e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0880b6c70 0x7fb088151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.959+0000 7fb08e000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0880d2b50 0x7fb0880ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.959+0000 7fb08e000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb08815ba60 0x7fb0881f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.959+0000 7fb08e000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb08815ff60 con 0x7fb08815ba60 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.959+0000 7fb08e000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb08815ca60 con 0x7fb0880b6c70 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.959+0000 7fb08e000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb0880d1770 con 0x7fb0880d2b50 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.959+0000 7fb08d600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb08815ba60 0x7fb0881f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.959+0000 7fb08d600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb08815ba60 0x7fb0881f4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38266/0 (socket says 172.21.15.49:38266) 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.959+0000 7fb08d600640 1 -- 172.21.15.49:0/1586620973 learned_addr learned my addr 172.21.15.49:0/1586620973 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.960+0000 7fb08cc00640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0880b6c70 0x7fb088151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.960+0000 7fb087e00640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0880d2b50 0x7fb0880ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.960+0000 7fb08d600640 1 -- 172.21.15.49:0/1586620973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0880d2b50 msgr2=0x7fb0880ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.960+0000 7fb08d600640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0880d2b50 0x7fb0880ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.960+0000 7fb08d600640 1 -- 172.21.15.49:0/1586620973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0880b6c70 msgr2=0x7fb088151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.960+0000 7fb08d600640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0880b6c70 0x7fb088151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.960+0000 7fb08d600640 1 -- 172.21.15.49:0/1586620973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0880c9700 con 0x7fb08815ba60 2024-04-15T15:19:35.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.960+0000 7fb08d600640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb08815ba60 0x7fb0881f4a70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb08000efc0 tx=0x7fb08000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.961+0000 7fb084c00640 1 -- 172.21.15.49:0/1586620973 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080018020 con 0x7fb08815ba60 2024-04-15T15:19:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.961+0000 7fb084c00640 1 -- 172.21.15.49:0/1586620973 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb080014480 con 0x7fb08815ba60 2024-04-15T15:19:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.961+0000 7fb084c00640 1 -- 172.21.15.49:0/1586620973 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080014870 con 0x7fb08815ba60 2024-04-15T15:19:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.961+0000 7fb08e000640 1 -- 172.21.15.49:0/1586620973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0880c99e0 con 0x7fb08815ba60 2024-04-15T15:19:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.961+0000 7fb08e000640 1 -- 172.21.15.49:0/1586620973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0880c9fa0 con 0x7fb08815ba60 2024-04-15T15:19:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.962+0000 7fb08e000640 1 -- 172.21.15.49:0/1586620973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0880755c0 con 0x7fb08815ba60 2024-04-15T15:19:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.963+0000 7fb084c00640 1 -- 172.21.15.49:0/1586620973 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb080005d10 con 0x7fb08815ba60 2024-04-15T15:19:35.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.964+0000 7fb084c00640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb06406c8f0 0x7fb06406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:35.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.964+0000 7fb084c00640 1 -- 172.21.15.49:0/1586620973 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fb08008c610 con 0x7fb08815ba60 2024-04-15T15:19:35.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.964+0000 7fb08cc00640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb06406c8f0 0x7fb06406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:35.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.965+0000 7fb08cc00640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb06406c8f0 0x7fb06406edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb078036000 tx=0x7fb078008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:35.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:35.968+0000 7fb084c00640 1 -- 172.21.15.49:0/1586620973 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb080022070 con 0x7fb08815ba60 2024-04-15T15:19:36.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.164+0000 7fb08e000640 1 -- 172.21.15.49:0/1586620973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb08806dbe0 con 0x7fb08815ba60 2024-04-15T15:19:36.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.165+0000 7fb084c00640 1 -- 172.21.15.49:0/1586620973 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fb080010040 con 0x7fb08815ba60 2024-04-15T15:19:36.166 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:36.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.167+0000 7fb08e000640 1 -- 172.21.15.49:0/1586620973 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb06406c8f0 msgr2=0x7fb06406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.167+0000 7fb08e000640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb06406c8f0 0x7fb06406edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb078036000 tx=0x7fb078008030 comp rx=0 tx=0).stop 2024-04-15T15:19:36.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.167+0000 7fb08e000640 1 -- 172.21.15.49:0/1586620973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb08815ba60 msgr2=0x7fb0881f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.167+0000 7fb08e000640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb08815ba60 0x7fb0881f4a70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb08000efc0 tx=0x7fb08000c490 comp rx=0 tx=0).stop 2024-04-15T15:19:36.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.167+0000 7fb08e000640 1 -- 172.21.15.49:0/1586620973 shutdown_connections 2024-04-15T15:19:36.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.167+0000 7fb08e000640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb06406c8f0 0x7fb06406edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.167+0000 7fb08e000640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0880d2b50 0x7fb0880ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.167+0000 7fb08e000640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb08815ba60 0x7fb0881f4a70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.167+0000 7fb08e000640 1 --2- 172.21.15.49:0/1586620973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0880b6c70 0x7fb088151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.167+0000 7fb08e000640 1 -- 172.21.15.49:0/1586620973 >> 172.21.15.49:0/1586620973 conn(0x7fb088073480 msgr2=0x7fb088073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:36.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.168+0000 7fb08e000640 1 -- 172.21.15.49:0/1586620973 shutdown_connections 2024-04-15T15:19:36.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.168+0000 7fb08e000640 1 -- 172.21.15.49:0/1586620973 wait complete. 2024-04-15T15:19:36.190 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:36.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.420+0000 7f10c5c00640 1 Processor -- start 2024-04-15T15:19:36.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.420+0000 7f10c5c00640 1 -- start start 2024-04-15T15:19:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.422+0000 7f10c5c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f10c01461c0 0x7f10c0071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.422+0000 7f10c5c00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10c00cd2d0 con 0x7f10c0072420 2024-04-15T15:19:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.422+0000 7f10c5c00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10c00cd5b0 con 0x7f10c00cebb0 2024-04-15T15:19:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.422+0000 7f10c5c00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f10c00cf1d0 con 0x7f10c01461c0 2024-04-15T15:19:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.423+0000 7f10b7e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f10c0072420 0x7f10c00ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37654/0 (socket says 172.21.15.49:37654) 2024-04-15T15:19:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.423+0000 7f10b7e00640 1 -- 172.21.15.49:0/3581216927 learned_addr learned my addr 172.21.15.49:0/3581216927 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:36.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.423+0000 7f10bfe00640 1 --2- 172.21.15.49:0/3581216927 >> v2:172.21.15.107:3300/0 conn(0x7f10c01461c0 0x7f10c0071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.423+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236511313 0 0) 0x7f10c00cd2d0 con 0x7f10c0072420 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.423+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10a4003520 con 0x7f10c0072420 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2436775174 0 0) 0x7f10a4003520 con 0x7f10c0072420 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 >> v2:172.21.15.107:3300/0 conn(0x7f10c01461c0 msgr2=0x7f10c0071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 --2- 172.21.15.49:0/3581216927 >> v2:172.21.15.107:3300/0 conn(0x7f10c01461c0 0x7f10c0071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 >> v1:172.21.15.107:6789/0 conn(0x7f10c00cebb0 legacy=0x7f10c006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c00cd890 con 0x7f10c0072420 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f10a8001670 con 0x7f10c0072420 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 >> v1:172.21.15.49:6789/0 conn(0x7f10c0072420 legacy=0x7f10c00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4003a00 0x7f10a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a4004800 0x7f10a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10a40051c0 0x7f10a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f10a400c090 con 0x7f10a4004800 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10a400c240 con 0x7f10a4003a00 2024-04-15T15:19:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f10a400c440 con 0x7f10a40051c0 2024-04-15T15:19:36.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10b7e00640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a4004800 0x7f10a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.424+0000 7f10bfe00640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4003a00 0x7f10a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.425+0000 7f10b7e00640 1 -- 172.21.15.49:0/3581216927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10a40051c0 msgr2=0x7f10a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:36.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.425+0000 7f10c5200640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10a40051c0 0x7f10a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.425+0000 7f10b7e00640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10a40051c0 0x7f10a400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.425+0000 7f10b7e00640 1 -- 172.21.15.49:0/3581216927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4003a00 msgr2=0x7f10a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:36.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.425+0000 7f10b7e00640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4003a00 0x7f10a4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.425+0000 7f10b7e00640 1 -- 172.21.15.49:0/3581216927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a4003520 con 0x7f10a4004800 2024-04-15T15:19:36.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.425+0000 7f10c5200640 1 -- 172.21.15.49:0/3581216927 reap_dead start 2024-04-15T15:19:36.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.426+0000 7f10b7e00640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a4004800 0x7f10a4004c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c190 tx=0x7f10a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c579df59f288f911 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.426+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8007c60 con 0x7f10a4004800 2024-04-15T15:19:36.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.426+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a8030070 con 0x7f10a4004800 2024-04-15T15:19:36.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.426+0000 7f10bf400640 1 -- 172.21.15.49:0/3581216927 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8005db0 con 0x7f10a4004800 2024-04-15T15:19:36.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.426+0000 7f10c5c00640 1 -- 172.21.15.49:0/3581216927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a4004800 msgr2=0x7f10a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.426+0000 7f10c5c00640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a4004800 0x7f10a4004c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c190 tx=0x7f10a4005830 comp rx=0 tx=0).stop 2024-04-15T15:19:36.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.427+0000 7f10c5c00640 1 -- 172.21.15.49:0/3581216927 shutdown_connections 2024-04-15T15:19:36.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.427+0000 7f10c5c00640 1 --2- 172.21.15.49:0/3581216927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a4004800 0x7f10a4004c80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.427+0000 7f10c5c00640 1 -- 172.21.15.49:0/3581216927 >> 172.21.15.49:0/3581216927 conn(0x7f10c006b930 msgr2=0x7f10c006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:36.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.427+0000 7f10c5c00640 1 -- 172.21.15.49:0/3581216927 shutdown_connections 2024-04-15T15:19:36.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.427+0000 7f10c5c00640 1 -- 172.21.15.49:0/3581216927 wait complete. 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.428+0000 7f10c5c00640 1 Processor -- start 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.429+0000 7f10c5c00640 1 -- start start 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.429+0000 7f10c5c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10a4004800 0x7f10c0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.429+0000 7f10c5c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10c015aaa0 0x7f10c015af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.429+0000 7f10c5c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c01e7b60 0x7f10c01e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.429+0000 7f10c5c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f10c00cd080 con 0x7f10c015aaa0 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.429+0000 7f10c5c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0071ee0 con 0x7f10c01e7b60 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.429+0000 7f10c5c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f10c0072060 con 0x7f10a4004800 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10b7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10c015aaa0 0x7f10c015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10b7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10c015aaa0 0x7f10c015af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38280/0 (socket says 172.21.15.49:38280) 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10b7e00640 1 -- 172.21.15.49:0/3001467637 learned_addr learned my addr 172.21.15.49:0/3001467637 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10bfe00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10a4004800 0x7f10c0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10c5200640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c01e7b60 0x7f10c01e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10b7e00640 1 -- 172.21.15.49:0/3001467637 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10a4004800 msgr2=0x7f10c0155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10b7e00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10a4004800 0x7f10c0155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10b7e00640 1 -- 172.21.15.49:0/3001467637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c01e7b60 msgr2=0x7f10c01e7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10b7e00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c01e7b60 0x7f10c01e7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10b7e00640 1 -- 172.21.15.49:0/3001467637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c01e8850 con 0x7f10c015aaa0 2024-04-15T15:19:36.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.430+0000 7f10bfe00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10a4004800 0x7f10c0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:36.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.431+0000 7f10b7e00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10c015aaa0 0x7f10c015af20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c2b0 tx=0x7f10a802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.431+0000 7f10bcc00640 1 -- 172.21.15.49:0/3001467637 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8046070 con 0x7f10c015aaa0 2024-04-15T15:19:36.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.431+0000 7f10bcc00640 1 -- 172.21.15.49:0/3001467637 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a8000ed0 con 0x7f10c015aaa0 2024-04-15T15:19:36.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.431+0000 7f10bcc00640 1 -- 172.21.15.49:0/3001467637 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8004e30 con 0x7f10c015aaa0 2024-04-15T15:19:36.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.431+0000 7f10c5c00640 1 -- 172.21.15.49:0/3001467637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c01e8b30 con 0x7f10c015aaa0 2024-04-15T15:19:36.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.431+0000 7f10c5c00640 1 -- 172.21.15.49:0/3001467637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c01584c0 con 0x7f10c015aaa0 2024-04-15T15:19:36.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.432+0000 7f10c5c00640 1 -- 172.21.15.49:0/3001467637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c00755c0 con 0x7f10c015aaa0 2024-04-15T15:19:36.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.433+0000 7f10bcc00640 1 -- 172.21.15.49:0/3001467637 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10a8007070 con 0x7f10c015aaa0 2024-04-15T15:19:36.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.434+0000 7f10bcc00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f109c06c8f0 0x7f109c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.434+0000 7f10bcc00640 1 -- 172.21.15.49:0/3001467637 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f10a80acfa0 con 0x7f10c015aaa0 2024-04-15T15:19:36.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.434+0000 7f10bfe00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f109c06c8f0 0x7f109c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.435+0000 7f10bfe00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f109c06c8f0 0x7f109c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009a80 tx=0x7f10b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.438+0000 7f10bcc00640 1 -- 172.21.15.49:0/3001467637 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10a807b2b0 con 0x7f10c015aaa0 2024-04-15T15:19:36.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.644+0000 7f10c5c00640 1 -- 172.21.15.49:0/3001467637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10c0068c80 con 0x7f10c015aaa0 2024-04-15T15:19:36.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.645+0000 7f10bcc00640 1 -- 172.21.15.49:0/3001467637 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f10a807abb0 con 0x7f10c015aaa0 2024-04-15T15:19:36.647 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:36.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.648+0000 7f10c5c00640 1 -- 172.21.15.49:0/3001467637 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f109c06c8f0 msgr2=0x7f109c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.648+0000 7f10c5c00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f109c06c8f0 0x7f109c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009a80 tx=0x7f10b0008040 comp rx=0 tx=0).stop 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.648+0000 7f10c5c00640 1 -- 172.21.15.49:0/3001467637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10c015aaa0 msgr2=0x7f10c015af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.648+0000 7f10c5c00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10c015aaa0 0x7f10c015af20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c2b0 tx=0x7f10a802e070 comp rx=0 tx=0).stop 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.649+0000 7f10c5c00640 1 -- 172.21.15.49:0/3001467637 shutdown_connections 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.649+0000 7f10c5c00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f109c06c8f0 0x7f109c06edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.649+0000 7f10c5c00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c01e7b60 0x7f10c01e7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.649+0000 7f10c5c00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10c015aaa0 0x7f10c015af20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.649+0000 7f10c5c00640 1 --2- 172.21.15.49:0/3001467637 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10a4004800 0x7f10c0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.649+0000 7f10c5c00640 1 -- 172.21.15.49:0/3001467637 >> 172.21.15.49:0/3001467637 conn(0x7f10c006b930 msgr2=0x7f10c015de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.649+0000 7f10c5c00640 1 -- 172.21.15.49:0/3001467637 shutdown_connections 2024-04-15T15:19:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.649+0000 7f10c5c00640 1 -- 172.21.15.49:0/3001467637 wait complete. 2024-04-15T15:19:36.671 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:19:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.899+0000 7f7fbca00640 1 Processor -- start 2024-04-15T15:19:36.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.899+0000 7f7fbca00640 1 -- start start 2024-04-15T15:19:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.901+0000 7f7fbca00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fb80b5b00 0x7f7fb80afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.901+0000 7f7fbca00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7fb8162dd0 0x7f7fb80b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.901+0000 7f7fbca00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7fb80b5530 0x7f7fb80b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.901+0000 7f7fbca00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7fb80b6420 con 0x7f7fb8162dd0 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.901+0000 7f7fbca00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f7fb80ce760 con 0x7f7fb80b5530 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.901+0000 7f7fbca00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7fb80cea40 con 0x7f7fb80b5b00 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.901+0000 7f7fb5e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7fb8162dd0 0x7f7fb80b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.901+0000 7f7fb5e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7fb8162dd0 0x7f7fb80b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38284/0 (socket says 172.21.15.49:38284) 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.901+0000 7f7fb5e00640 1 -- 172.21.15.49:0/1206647131 learned_addr learned my addr 172.21.15.49:0/1206647131 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.901+0000 7f7fb6800640 1 --2- 172.21.15.49:0/1206647131 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fb80b5b00 0x7f7fb80afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.902+0000 7f7fb7200640 1 --2- 172.21.15.49:0/1206647131 >> v2:172.21.15.107:3300/0 conn(0x7f7fb80b5530 0x7f7fb80b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.902+0000 7f7fb5e00640 1 -- 172.21.15.49:0/1206647131 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fb80b5b00 msgr2=0x7f7fb80afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:36.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.902+0000 7f7fb5e00640 1 --2- 172.21.15.49:0/1206647131 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fb80b5b00 0x7f7fb80afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.902+0000 7f7fb5e00640 1 -- 172.21.15.49:0/1206647131 >> v2:172.21.15.107:3300/0 conn(0x7f7fb80b5530 msgr2=0x7f7fb80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.902+0000 7f7fb5e00640 1 --2- 172.21.15.49:0/1206647131 >> v2:172.21.15.107:3300/0 conn(0x7f7fb80b5530 0x7f7fb80b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.902+0000 7f7fb5e00640 1 -- 172.21.15.49:0/1206647131 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb80ced20 con 0x7f7fb8162dd0 2024-04-15T15:19:36.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.902+0000 7f7fb7200640 1 --2- 172.21.15.49:0/1206647131 >> v2:172.21.15.107:3300/0 conn(0x7f7fb80b5530 0x7f7fb80b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:36.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.903+0000 7f7fb5e00640 1 --2- 172.21.15.49:0/1206647131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb8162dd0 0x7f7fb80b5ee0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8073580 tx=0x7f7fa402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69979aa71a2c6fd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.903+0000 7f7fb5400640 1 -- 172.21.15.49:0/1206647131 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4035020 con 0x7f7fb8162dd0 2024-04-15T15:19:36.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.903+0000 7f7fb5400640 1 -- 172.21.15.49:0/1206647131 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa4031880 con 0x7f7fb8162dd0 2024-04-15T15:19:36.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.903+0000 7f7fb5400640 1 -- 172.21.15.49:0/1206647131 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4031c50 con 0x7f7fb8162dd0 2024-04-15T15:19:36.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.904+0000 7f7fbca00640 1 -- 172.21.15.49:0/1206647131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb8162dd0 msgr2=0x7f7fb80b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.904+0000 7f7fbca00640 1 --2- 172.21.15.49:0/1206647131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb8162dd0 0x7f7fb80b5ee0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8073580 tx=0x7f7fa402b920 comp rx=0 tx=0).stop 2024-04-15T15:19:36.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.904+0000 7f7fbca00640 1 -- 172.21.15.49:0/1206647131 shutdown_connections 2024-04-15T15:19:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.904+0000 7f7fbca00640 1 --2- 172.21.15.49:0/1206647131 >> v2:172.21.15.107:3300/0 conn(0x7f7fb80b5530 0x7f7fb80b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.904+0000 7f7fbca00640 1 --2- 172.21.15.49:0/1206647131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb8162dd0 0x7f7fb80b5ee0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.904+0000 7f7fbca00640 1 --2- 172.21.15.49:0/1206647131 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fb80b5b00 0x7f7fb80afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.904+0000 7f7fbca00640 1 -- 172.21.15.49:0/1206647131 >> 172.21.15.49:0/1206647131 conn(0x7f7fb80b23a0 msgr2=0x7f7fb80b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.904+0000 7f7fbca00640 1 -- 172.21.15.49:0/1206647131 shutdown_connections 2024-04-15T15:19:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.904+0000 7f7fbca00640 1 -- 172.21.15.49:0/1206647131 wait complete. 2024-04-15T15:19:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.906+0000 7f7fbca00640 1 Processor -- start 2024-04-15T15:19:36.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.906+0000 7f7fbca00640 1 -- start start 2024-04-15T15:19:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.907+0000 7f7fbca00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fb80b5530 0x7f7fb814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.907+0000 7f7fbca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb80b5b00 0x7f7fb81585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.907+0000 7f7fbca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fb8162dd0 0x7f7fb815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.907+0000 7f7fbca00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb80cf520 con 0x7f7fb80b5b00 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.907+0000 7f7fbca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb80ce410 con 0x7f7fb8162dd0 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.907+0000 7f7fbca00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7fb80cf6a0 con 0x7f7fb80b5530 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.907+0000 7f7fb5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb80b5b00 0x7f7fb81585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.907+0000 7f7fb5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb80b5b00 0x7f7fb81585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38294/0 (socket says 172.21.15.49:38294) 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.907+0000 7f7fb5e00640 1 -- 172.21.15.49:0/1945040819 learned_addr learned my addr 172.21.15.49:0/1945040819 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.908+0000 7f7fb6800640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fb80b5530 0x7f7fb814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.908+0000 7f7fb7200640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fb8162dd0 0x7f7fb815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.908+0000 7f7fb5e00640 1 -- 172.21.15.49:0/1945040819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fb80b5530 msgr2=0x7f7fb814d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.908+0000 7f7fb5e00640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fb80b5530 0x7f7fb814d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.908+0000 7f7fb5e00640 1 -- 172.21.15.49:0/1945040819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fb8162dd0 msgr2=0x7f7fb815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:36.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.908+0000 7f7fb5e00640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fb8162dd0 0x7f7fb815d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:36.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.908+0000 7f7fb5e00640 1 -- 172.21.15.49:0/1945040819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb814de00 con 0x7f7fb80b5b00 2024-04-15T15:19:36.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.908+0000 7f7fb7200640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fb8162dd0 0x7f7fb815d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:36.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.908+0000 7f7fb5e00640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb80b5b00 0x7f7fb81585d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7fa402b960 tx=0x7f7fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.909+0000 7f7faea00640 1 -- 172.21.15.49:0/1945040819 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4040070 con 0x7f7fb80b5b00 2024-04-15T15:19:36.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.909+0000 7f7faea00640 1 -- 172.21.15.49:0/1945040819 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa4030c10 con 0x7f7fb80b5b00 2024-04-15T15:19:36.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.909+0000 7f7faea00640 1 -- 172.21.15.49:0/1945040819 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4035040 con 0x7f7fb80b5b00 2024-04-15T15:19:36.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.909+0000 7f7fbca00640 1 -- 172.21.15.49:0/1945040819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb815a5b0 con 0x7f7fb80b5b00 2024-04-15T15:19:36.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.909+0000 7f7fbca00640 1 -- 172.21.15.49:0/1945040819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb815a210 con 0x7f7fb80b5b00 2024-04-15T15:19:36.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.910+0000 7f7fbca00640 1 -- 172.21.15.49:0/1945040819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb815cc40 con 0x7f7fb80b5b00 2024-04-15T15:19:36.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.911+0000 7f7faea00640 1 -- 172.21.15.49:0/1945040819 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fa404b060 con 0x7f7fb80b5b00 2024-04-15T15:19:36.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.912+0000 7f7faea00640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7f9806c8f0 0x7f7f9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:36.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.912+0000 7f7faea00640 1 -- 172.21.15.49:0/1945040819 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f7fa40adaf0 con 0x7f7fb80b5b00 2024-04-15T15:19:36.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.912+0000 7f7fb6800640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7f9806c8f0 0x7f7f9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:36.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.913+0000 7f7fb6800640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7f9806c8f0 0x7f7f9806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8006fd0 tx=0x7f7fa8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:36.917+0000 7f7faea00640 1 -- 172.21.15.49:0/1945040819 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fb814de00 con 0x7f7fb80b5b00 2024-04-15T15:19:37.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.137+0000 7f7fbca00640 1 -- 172.21.15.49:0/1945040819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7fb815aad0 con 0x7f7fb80b5b00 2024-04-15T15:19:37.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.137+0000 7f7faea00640 1 -- 172.21.15.49:0/1945040819 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f7fa404b350 con 0x7f7fb80b5b00 2024-04-15T15:19:37.139 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2024-04-15T15:19:37.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.140+0000 7f7fbca00640 1 -- 172.21.15.49:0/1945040819 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7f9806c8f0 msgr2=0x7f7f9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.140+0000 7f7fbca00640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7f9806c8f0 0x7f7f9806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8006fd0 tx=0x7f7fa8008040 comp rx=0 tx=0).stop 2024-04-15T15:19:37.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.140+0000 7f7fbca00640 1 -- 172.21.15.49:0/1945040819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb80b5b00 msgr2=0x7f7fb81585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.140+0000 7f7fbca00640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb80b5b00 0x7f7fb81585d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7fa402b960 tx=0x7f7fa40046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.141+0000 7f7fbca00640 1 -- 172.21.15.49:0/1945040819 shutdown_connections 2024-04-15T15:19:37.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.141+0000 7f7fbca00640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7f9806c8f0 0x7f7f9806edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.141+0000 7f7fbca00640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fb8162dd0 0x7f7fb815d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.141+0000 7f7fbca00640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fb80b5b00 0x7f7fb81585d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.141+0000 7f7fbca00640 1 --2- 172.21.15.49:0/1945040819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fb80b5530 0x7f7fb814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.141+0000 7f7fbca00640 1 -- 172.21.15.49:0/1945040819 >> 172.21.15.49:0/1945040819 conn(0x7f7fb80b23a0 msgr2=0x7f7fb80cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:37.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.141+0000 7f7fbca00640 1 -- 172.21.15.49:0/1945040819 shutdown_connections 2024-04-15T15:19:37.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.141+0000 7f7fbca00640 1 -- 172.21.15.49:0/1945040819 wait complete. 2024-04-15T15:19:37.163 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-15T15:19:37.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:37.163 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-15T15:19:37.268 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:19:37.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:37.268 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-15T15:19:37.323 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:19:37.323 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:37.384 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:19:37.385 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:37.440 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:19:37.441 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:37.456 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:19:37.494 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:19:37.497 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:37.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.763+0000 7fd047200640 1 Processor -- start 2024-04-15T15:19:37.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.763+0000 7fd047200640 1 -- start start 2024-04-15T15:19:37.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.765+0000 7fd047200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd04815b480 0x7fd0480cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.765+0000 7fd047200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd0480cdde0 0x7fd04806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.765+0000 7fd047200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd04815b860 0x7fd04806f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.765+0000 7fd047200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd04815f060 con 0x7fd0480cdde0 2024-04-15T15:19:37.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.765+0000 7fd047200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fd04815fec0 con 0x7fd04815b480 2024-04-15T15:19:37.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.765+0000 7fd047200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd04815f2b0 con 0x7fd04815b860 2024-04-15T15:19:37.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.766+0000 7fd045400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd0480cdde0 0x7fd04806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.766+0000 7fd045400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd0480cdde0 0x7fd04806ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38310/0 (socket says 172.21.15.49:38310) 2024-04-15T15:19:37.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.766+0000 7fd045400640 1 -- 172.21.15.49:0/1300431585 learned_addr learned my addr 172.21.15.49:0/1300431585 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:37.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.766+0000 7fd045e00640 1 --2- 172.21.15.49:0/1300431585 >> v2:172.21.15.107:3300/0 conn(0x7fd04815b480 0x7fd0480cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.766+0000 7fd045400640 1 -- 172.21.15.49:0/1300431585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd04815b860 msgr2=0x7fd04806f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:37.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.766+0000 7fd046800640 1 --2- 172.21.15.49:0/1300431585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd04815b860 0x7fd04806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.766+0000 7fd045400640 1 --2- 172.21.15.49:0/1300431585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd04815b860 0x7fd04806f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.766+0000 7fd045400640 1 -- 172.21.15.49:0/1300431585 >> v2:172.21.15.107:3300/0 conn(0x7fd04815b480 msgr2=0x7fd0480cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:37.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.766+0000 7fd045400640 1 --2- 172.21.15.49:0/1300431585 >> v2:172.21.15.107:3300/0 conn(0x7fd04815b480 0x7fd0480cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.766+0000 7fd045400640 1 -- 172.21.15.49:0/1300431585 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04815f590 con 0x7fd0480cdde0 2024-04-15T15:19:37.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.767+0000 7fd045400640 1 --2- 172.21.15.49:0/1300431585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 0x7fd04806ede0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd04806c720 tx=0x7fd03802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b4e1ab5fc7086 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.768+0000 7fd044a00640 1 -- 172.21.15.49:0/1300431585 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03802ea80 con 0x7fd0480cdde0 2024-04-15T15:19:37.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.768+0000 7fd044a00640 1 -- 172.21.15.49:0/1300431585 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd03802ec20 con 0x7fd0480cdde0 2024-04-15T15:19:37.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.768+0000 7fd044a00640 1 -- 172.21.15.49:0/1300431585 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038009600 con 0x7fd0480cdde0 2024-04-15T15:19:37.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.768+0000 7fd047200640 1 -- 172.21.15.49:0/1300431585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 msgr2=0x7fd04806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.768+0000 7fd047200640 1 --2- 172.21.15.49:0/1300431585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 0x7fd04806ede0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd04806c720 tx=0x7fd03802e0e0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.768+0000 7fd047200640 1 -- 172.21.15.49:0/1300431585 shutdown_connections 2024-04-15T15:19:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.768+0000 7fd047200640 1 --2- 172.21.15.49:0/1300431585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd04815b860 0x7fd04806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.768+0000 7fd047200640 1 --2- 172.21.15.49:0/1300431585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 0x7fd04806ede0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.769+0000 7fd047200640 1 --2- 172.21.15.49:0/1300431585 >> v2:172.21.15.107:3300/0 conn(0x7fd04815b480 0x7fd0480cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.769+0000 7fd047200640 1 -- 172.21.15.49:0/1300431585 >> 172.21.15.49:0/1300431585 conn(0x7fd04806b930 msgr2=0x7fd04806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.769+0000 7fd047200640 1 -- 172.21.15.49:0/1300431585 shutdown_connections 2024-04-15T15:19:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.769+0000 7fd047200640 1 -- 172.21.15.49:0/1300431585 wait complete. 2024-04-15T15:19:37.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.770+0000 7fd047200640 1 Processor -- start 2024-04-15T15:19:37.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.771+0000 7fd047200640 1 -- start start 2024-04-15T15:19:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.771+0000 7fd047200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 0x7fd0481490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.771+0000 7fd047200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd04815b480 0x7fd0480c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd047200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd04815b860 0x7fd0480c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd047200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0480cd550 con 0x7fd0480cdde0 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd047200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd04815fb50 con 0x7fd04815b860 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd047200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd048071f50 con 0x7fd04815b480 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd045e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 0x7fd0481490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd045e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 0x7fd0481490d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38324/0 (socket says 172.21.15.49:38324) 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd045e00640 1 -- 172.21.15.49:0/4243588405 learned_addr learned my addr 172.21.15.49:0/4243588405 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd045400640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd04815b480 0x7fd0480c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd046800640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd04815b860 0x7fd0480c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd045e00640 1 -- 172.21.15.49:0/4243588405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd04815b480 msgr2=0x7fd0480c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd045e00640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd04815b480 0x7fd0480c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd045e00640 1 -- 172.21.15.49:0/4243588405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd04815b860 msgr2=0x7fd0480c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd045e00640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd04815b860 0x7fd0480c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.772+0000 7fd045e00640 1 -- 172.21.15.49:0/4243588405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0480c6970 con 0x7fd0480cdde0 2024-04-15T15:19:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.773+0000 7fd045e00640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 0x7fd0481490d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd03000d800 tx=0x7fd03000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.773+0000 7fd03e000640 1 -- 172.21.15.49:0/4243588405 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03000b660 con 0x7fd0480cdde0 2024-04-15T15:19:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.773+0000 7fd03e000640 1 -- 172.21.15.49:0/4243588405 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd030019070 con 0x7fd0480cdde0 2024-04-15T15:19:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.773+0000 7fd03e000640 1 -- 172.21.15.49:0/4243588405 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03000ba20 con 0x7fd0480cdde0 2024-04-15T15:19:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.773+0000 7fd047200640 1 -- 172.21.15.49:0/4243588405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0480c6c50 con 0x7fd0480cdde0 2024-04-15T15:19:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.773+0000 7fd047200640 1 -- 172.21.15.49:0/4243588405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0481581b0 con 0x7fd0480cdde0 2024-04-15T15:19:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.774+0000 7fd047200640 1 -- 172.21.15.49:0/4243588405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0480755c0 con 0x7fd0480cdde0 2024-04-15T15:19:37.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.776+0000 7fd03e000640 1 -- 172.21.15.49:0/4243588405 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd03000bbc0 con 0x7fd0480cdde0 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.776+0000 7fd03e000640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd02406c8f0 0x7fd02406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.777+0000 7fd03e000640 1 -- 172.21.15.49:0/4243588405 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fd03008c080 con 0x7fd0480cdde0 2024-04-15T15:19:37.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.777+0000 7fd045400640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd02406c8f0 0x7fd02406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:37.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.778+0000 7fd045400640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd02406c8f0 0x7fd02406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd038037000 tx=0x7fd038032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:37.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.781+0000 7fd03e000640 1 -- 172.21.15.49:0/4243588405 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd03005a430 con 0x7fd0480cdde0 2024-04-15T15:19:37.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.988+0000 7fd047200640 1 -- 172.21.15.49:0/4243588405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd048156540 con 0x7fd0480cdde0 2024-04-15T15:19:37.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.989+0000 7fd03e000640 1 -- 172.21.15.49:0/4243588405 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16854 (secure 0 0 0) 0x7fd030059d30 con 0x7fd0480cdde0 2024-04-15T15:19:37.992 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:37.992 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:19:32.083681+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 -- 172.21.15.49:0/4243588405 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd02406c8f0 msgr2=0x7fd02406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd02406c8f0 0x7fd02406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd038037000 tx=0x7fd038032000 comp rx=0 tx=0).stop 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 -- 172.21.15.49:0/4243588405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 msgr2=0x7fd0481490d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 0x7fd0481490d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd03000d800 tx=0x7fd03000dcc0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 -- 172.21.15.49:0/4243588405 shutdown_connections 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd02406c8f0 0x7fd02406edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd04815b480 0x7fd0480c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd04815b860 0x7fd0480c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 --2- 172.21.15.49:0/4243588405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0480cdde0 0x7fd0481490d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 -- 172.21.15.49:0/4243588405 >> 172.21.15.49:0/4243588405 conn(0x7fd04806b930 msgr2=0x7fd04815e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 -- 172.21.15.49:0/4243588405 shutdown_connections 2024-04-15T15:19:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:37.993+0000 7fd047200640 1 -- 172.21.15.49:0/4243588405 wait complete. 2024-04-15T15:19:38.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:38.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.250+0000 7f0e97e00640 1 Processor -- start 2024-04-15T15:19:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.250+0000 7f0e97e00640 1 -- start start 2024-04-15T15:19:38.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.251+0000 7f0e97e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0e980d1580 0x7f0e980d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.251+0000 7f0e97e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e980d1ef0 con 0x7f0e980d26b0 2024-04-15T15:19:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.252+0000 7f0e97e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e980cfef0 con 0x7f0e98161f80 2024-04-15T15:19:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.252+0000 7f0e97e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f0e980d2bb0 con 0x7f0e980d1580 2024-04-15T15:19:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.252+0000 7f0e96000640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0e980d26b0 0x7f0e980b7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37658/0 (socket says 172.21.15.49:37658) 2024-04-15T15:19:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.252+0000 7f0e96000640 1 -- 172.21.15.49:0/1486876925 learned_addr learned my addr 172.21.15.49:0/1486876925 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.252+0000 7f0e97400640 1 --2- 172.21.15.49:0/1486876925 >> v2:172.21.15.107:3300/0 conn(0x7f0e980d1580 0x7f0e980d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.252+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437354360 0 0) 0x7f0e980d1ef0 con 0x7f0e980d26b0 2024-04-15T15:19:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.252+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e78003520 con 0x7f0e980d26b0 2024-04-15T15:19:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1474035317 0 0) 0x7f0e980cfef0 con 0x7f0e98161f80 2024-04-15T15:19:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e980d1ef0 con 0x7f0e98161f80 2024-04-15T15:19:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 164443742 0 0) 0x7f0e78003520 con 0x7f0e980d26b0 2024-04-15T15:19:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 >> v2:172.21.15.107:3300/0 conn(0x7f0e980d1580 msgr2=0x7f0e980d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 --2- 172.21.15.49:0/1486876925 >> v2:172.21.15.107:3300/0 conn(0x7f0e980d1580 0x7f0e980d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 >> v1:172.21.15.107:6789/0 conn(0x7f0e98161f80 legacy=0x7f0e980cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e980d2e90 con 0x7f0e980d26b0 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f0e80001670 con 0x7f0e980d26b0 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 >> v1:172.21.15.49:6789/0 conn(0x7f0e980d26b0 legacy=0x7f0e980b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e78003b90 0x7f0e78003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e78004800 0x7f0e78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e780051c0 0x7f0e7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0e980d2bb0 con 0x7f0e780051c0 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7800c090 con 0x7f0e78003b90 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0e7800c290 con 0x7f0e78004800 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e97400640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e780051c0 0x7f0e7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.253+0000 7f0e96a00640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e78003b90 0x7f0e78003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.254+0000 7f0e96000640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e78004800 0x7f0e78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.254+0000 7f0e97400640 1 -- 172.21.15.49:0/1486876925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e78004800 msgr2=0x7f0e78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.254+0000 7f0e97400640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e78004800 0x7f0e78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.254+0000 7f0e97400640 1 -- 172.21.15.49:0/1486876925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e78003b90 msgr2=0x7f0e78003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.254+0000 7f0e97400640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e78003b90 0x7f0e78003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.254+0000 7f0e97400640 1 -- 172.21.15.49:0/1486876925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e78003520 con 0x7f0e780051c0 2024-04-15T15:19:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.255+0000 7f0e97400640 1 -- 172.21.15.49:0/1486876925 reap_dead start 2024-04-15T15:19:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.255+0000 7f0e97400640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e780051c0 0x7f0e7800b9c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009fc0 tx=0x7f0e78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6748eca42dc2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.255+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90009c10 con 0x7f0e780051c0 2024-04-15T15:19:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.255+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9002cca0 con 0x7f0e780051c0 2024-04-15T15:19:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.256+0000 7f0e95600640 1 -- 172.21.15.49:0/1486876925 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90003bd0 con 0x7f0e780051c0 2024-04-15T15:19:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.256+0000 7f0e97e00640 1 -- 172.21.15.49:0/1486876925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e780051c0 msgr2=0x7f0e7800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.256+0000 7f0e97e00640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e780051c0 0x7f0e7800b9c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009fc0 tx=0x7f0e78005830 comp rx=0 tx=0).stop 2024-04-15T15:19:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.256+0000 7f0e97e00640 1 -- 172.21.15.49:0/1486876925 shutdown_connections 2024-04-15T15:19:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.256+0000 7f0e97e00640 1 --2- 172.21.15.49:0/1486876925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e780051c0 0x7f0e7800b9c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.256+0000 7f0e97e00640 1 -- 172.21.15.49:0/1486876925 >> 172.21.15.49:0/1486876925 conn(0x7f0e9806ce40 msgr2=0x7f0e9806d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.256+0000 7f0e97e00640 1 -- 172.21.15.49:0/1486876925 shutdown_connections 2024-04-15T15:19:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.256+0000 7f0e97e00640 1 -- 172.21.15.49:0/1486876925 wait complete. 2024-04-15T15:19:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.258+0000 7f0e97e00640 1 Processor -- start 2024-04-15T15:19:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.258+0000 7f0e97e00640 1 -- start start 2024-04-15T15:19:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.259+0000 7f0e97e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e780051c0 0x7f0e980d4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.259+0000 7f0e97e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e980ca0a0 0x7f0e980d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.259+0000 7f0e97e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e980ca740 0x7f0e980cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.259+0000 7f0e97e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98162360 con 0x7f0e980ca740 2024-04-15T15:19:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.259+0000 7f0e97e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0e980d3450 con 0x7f0e780051c0 2024-04-15T15:19:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.259+0000 7f0e97e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0e980d19b0 con 0x7f0e980ca0a0 2024-04-15T15:19:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.260+0000 7f0e97400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e980ca740 0x7f0e980cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.260+0000 7f0e97400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e980ca740 0x7f0e980cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38342/0 (socket says 172.21.15.49:38342) 2024-04-15T15:19:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.260+0000 7f0e97400640 1 -- 172.21.15.49:0/956388978 learned_addr learned my addr 172.21.15.49:0/956388978 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.260+0000 7f0e96a00640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e780051c0 0x7f0e980d4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.260+0000 7f0e96000640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e980ca0a0 0x7f0e980d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.260+0000 7f0e97400640 1 -- 172.21.15.49:0/956388978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e980ca0a0 msgr2=0x7f0e980d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.260+0000 7f0e97400640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e980ca0a0 0x7f0e980d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.260+0000 7f0e97400640 1 -- 172.21.15.49:0/956388978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e780051c0 msgr2=0x7f0e980d4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.260+0000 7f0e97400640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e780051c0 0x7f0e980d4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.260+0000 7f0e97400640 1 -- 172.21.15.49:0/956388978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e980c8690 con 0x7f0e980ca740 2024-04-15T15:19:38.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.261+0000 7f0e97400640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e980ca740 0x7f0e980cabc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0e9002c6d0 tx=0x7f0e90009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.261+0000 7f0e8ea00640 1 -- 172.21.15.49:0/956388978 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90009c10 con 0x7f0e980ca740 2024-04-15T15:19:38.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.261+0000 7f0e8ea00640 1 -- 172.21.15.49:0/956388978 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e90044070 con 0x7f0e980ca740 2024-04-15T15:19:38.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.261+0000 7f0e8ea00640 1 -- 172.21.15.49:0/956388978 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90003b90 con 0x7f0e980ca740 2024-04-15T15:19:38.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.261+0000 7f0e97e00640 1 -- 172.21.15.49:0/956388978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e980c8970 con 0x7f0e980ca740 2024-04-15T15:19:38.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.261+0000 7f0e97e00640 1 -- 172.21.15.49:0/956388978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e980c8f30 con 0x7f0e980ca740 2024-04-15T15:19:38.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.262+0000 7f0e97e00640 1 -- 172.21.15.49:0/956388978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e980755c0 con 0x7f0e980ca740 2024-04-15T15:19:38.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.263+0000 7f0e8ea00640 1 -- 172.21.15.49:0/956388978 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e90005d10 con 0x7f0e980ca740 2024-04-15T15:19:38.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.265+0000 7f0e8ea00640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7406c920 0x7f0e7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.265+0000 7f0e8ea00640 1 -- 172.21.15.49:0/956388978 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f0e900ae410 con 0x7f0e980ca740 2024-04-15T15:19:38.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.265+0000 7f0e96a00640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7406c920 0x7f0e7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.266+0000 7f0e96a00640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7406c920 0x7f0e7406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e880018b0 tx=0x7f0e88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.270+0000 7f0e8ea00640 1 -- 172.21.15.49:0/956388978 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e900333a0 con 0x7f0e980ca740 2024-04-15T15:19:38.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.475+0000 7f0e97e00640 1 -- 172.21.15.49:0/956388978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0e98068ef0 con 0x7f0e980ca740 2024-04-15T15:19:38.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.476+0000 7f0e8ea00640 1 -- 172.21.15.49:0/956388978 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f0e9007c840 con 0x7f0e980ca740 2024-04-15T15:19:38.478 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:38.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 -- 172.21.15.49:0/956388978 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7406c920 msgr2=0x7f0e7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7406c920 0x7f0e7406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e880018b0 tx=0x7f0e88009040 comp rx=0 tx=0).stop 2024-04-15T15:19:38.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 -- 172.21.15.49:0/956388978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e980ca740 msgr2=0x7f0e980cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e980ca740 0x7f0e980cabc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0e9002c6d0 tx=0x7f0e90009750 comp rx=0 tx=0).stop 2024-04-15T15:19:38.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 -- 172.21.15.49:0/956388978 shutdown_connections 2024-04-15T15:19:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0e7406c920 0x7f0e7406ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e980ca0a0 0x7f0e980d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e980ca740 0x7f0e980cabc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 --2- 172.21.15.49:0/956388978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e780051c0 0x7f0e980d4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 -- 172.21.15.49:0/956388978 >> 172.21.15.49:0/956388978 conn(0x7f0e9806ce40 msgr2=0x7f0e980cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.479+0000 7f0e97e00640 1 -- 172.21.15.49:0/956388978 shutdown_connections 2024-04-15T15:19:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.480+0000 7f0e97e00640 1 -- 172.21.15.49:0/956388978 wait complete. 2024-04-15T15:19:38.502 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:38.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.737+0000 7fab90a00640 1 Processor -- start 2024-04-15T15:19:38.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.737+0000 7fab90a00640 1 -- start start 2024-04-15T15:19:38.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.738+0000 7fab90a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fab8c15c680 0x7fab8c162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.738+0000 7fab90a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 0x7fab8c15f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.738+0000 7fab90a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fab8c15fd90 0x7fab8c0d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.738+0000 7fab90a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fab8c0b70d0 con 0x7fab8c15c680 2024-04-15T15:19:38.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.738+0000 7fab90a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fab8c162dd0 con 0x7fab8c15fd90 2024-04-15T15:19:38.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.738+0000 7fab90a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c1630b0 con 0x7fab8c0b6c70 2024-04-15T15:19:38.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.739+0000 7fab8b400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fab8c15c680 0x7fab8c162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.739+0000 7fab8aa00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 0x7fab8c15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.739+0000 7fab8be00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fab8c15fd90 0x7fab8c0d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.739+0000 7fab8aa00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 0x7fab8c15f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:34716/0 (socket says 172.21.15.49:34716) 2024-04-15T15:19:38.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.739+0000 7fab8aa00640 1 -- 172.21.15.49:0/4050447554 learned_addr learned my addr 172.21.15.49:0/4050447554 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:38.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.740+0000 7fab8b400640 1 -- 172.21.15.49:0/4050447554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 msgr2=0x7fab8c15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.740+0000 7fab8b400640 1 --2- 172.21.15.49:0/4050447554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 0x7fab8c15f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.740+0000 7fab8b400640 1 -- 172.21.15.49:0/4050447554 >> v2:172.21.15.107:3300/0 conn(0x7fab8c15fd90 msgr2=0x7fab8c0d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:38.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.740+0000 7fab8b400640 1 --2- 172.21.15.49:0/4050447554 >> v2:172.21.15.107:3300/0 conn(0x7fab8c15fd90 0x7fab8c0d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.740+0000 7fab8b400640 1 -- 172.21.15.49:0/4050447554 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c163390 con 0x7fab8c15c680 2024-04-15T15:19:38.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.740+0000 7fab8aa00640 1 --2- 172.21.15.49:0/4050447554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 0x7fab8c15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:38.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.740+0000 7fab8be00640 1 --2- 172.21.15.49:0/4050447554 >> v2:172.21.15.107:3300/0 conn(0x7fab8c15fd90 0x7fab8c0d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:38.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.742+0000 7fab8b400640 1 --2- 172.21.15.49:0/4050447554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15c680 0x7fab8c162730 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0ba1f0 tx=0x7fab7c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d8fa2cb0bd6a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.742+0000 7fab8a000640 1 -- 172.21.15.49:0/4050447554 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0315a0 con 0x7fab8c15c680 2024-04-15T15:19:38.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.742+0000 7fab8a000640 1 -- 172.21.15.49:0/4050447554 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7c037070 con 0x7fab8c15c680 2024-04-15T15:19:38.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.742+0000 7fab8a000640 1 -- 172.21.15.49:0/4050447554 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c031960 con 0x7fab8c15c680 2024-04-15T15:19:38.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.743+0000 7fab90a00640 1 -- 172.21.15.49:0/4050447554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15c680 msgr2=0x7fab8c162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.743+0000 7fab90a00640 1 --2- 172.21.15.49:0/4050447554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15c680 0x7fab8c162730 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0ba1f0 tx=0x7fab7c02b840 comp rx=0 tx=0).stop 2024-04-15T15:19:38.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.743+0000 7fab90a00640 1 -- 172.21.15.49:0/4050447554 shutdown_connections 2024-04-15T15:19:38.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.743+0000 7fab90a00640 1 --2- 172.21.15.49:0/4050447554 >> v2:172.21.15.107:3300/0 conn(0x7fab8c15fd90 0x7fab8c0d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.743+0000 7fab90a00640 1 --2- 172.21.15.49:0/4050447554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 0x7fab8c15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.743+0000 7fab90a00640 1 --2- 172.21.15.49:0/4050447554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15c680 0x7fab8c162730 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.743+0000 7fab90a00640 1 -- 172.21.15.49:0/4050447554 >> 172.21.15.49:0/4050447554 conn(0x7fab8c073480 msgr2=0x7fab8c073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.743+0000 7fab90a00640 1 -- 172.21.15.49:0/4050447554 shutdown_connections 2024-04-15T15:19:38.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.743+0000 7fab90a00640 1 -- 172.21.15.49:0/4050447554 wait complete. 2024-04-15T15:19:38.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.745+0000 7fab90a00640 1 Processor -- start 2024-04-15T15:19:38.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.745+0000 7fab90a00640 1 -- start start 2024-04-15T15:19:38.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.746+0000 7fab90a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 0x7fab8c1f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.746+0000 7fab90a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab8c15c680 0x7fab8c0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.746+0000 7fab90a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15fd90 0x7fab8c0b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.746+0000 7fab90a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c0d3090 con 0x7fab8c15fd90 2024-04-15T15:19:38.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.746+0000 7fab90a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c15ca60 con 0x7fab8c15c680 2024-04-15T15:19:38.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.746+0000 7fab90a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c0d3210 con 0x7fab8c0b6c70 2024-04-15T15:19:38.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.746+0000 7fab8be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15fd90 0x7fab8c0b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.746+0000 7fab8be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15fd90 0x7fab8c0b8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38360/0 (socket says 172.21.15.49:38360) 2024-04-15T15:19:38.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.746+0000 7fab8be00640 1 -- 172.21.15.49:0/111310438 learned_addr learned my addr 172.21.15.49:0/111310438 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:38.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.747+0000 7fab8b400640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 0x7fab8c1f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.747+0000 7fab8aa00640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab8c15c680 0x7fab8c0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.747+0000 7fab8be00640 1 -- 172.21.15.49:0/111310438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 msgr2=0x7fab8c1f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.747+0000 7fab8be00640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 0x7fab8c1f1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.747+0000 7fab8be00640 1 -- 172.21.15.49:0/111310438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab8c15c680 msgr2=0x7fab8c0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.747+0000 7fab8be00640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab8c15c680 0x7fab8c0cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.747+0000 7fab8be00640 1 -- 172.21.15.49:0/111310438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c1f4a50 con 0x7fab8c15fd90 2024-04-15T15:19:38.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.747+0000 7fab8be00640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15fd90 0x7fab8c0b8b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fab8400efc0 tx=0x7fab8400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.748+0000 7fab83400640 1 -- 172.21.15.49:0/111310438 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84018020 con 0x7fab8c15fd90 2024-04-15T15:19:38.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.748+0000 7fab83400640 1 -- 172.21.15.49:0/111310438 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab84014480 con 0x7fab8c15fd90 2024-04-15T15:19:38.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.748+0000 7fab83400640 1 -- 172.21.15.49:0/111310438 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84014870 con 0x7fab8c15fd90 2024-04-15T15:19:38.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.748+0000 7fab90a00640 1 -- 172.21.15.49:0/111310438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c1f4d30 con 0x7fab8c15fd90 2024-04-15T15:19:38.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.748+0000 7fab90a00640 1 -- 172.21.15.49:0/111310438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1f52f0 con 0x7fab8c15fd90 2024-04-15T15:19:38.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.749+0000 7fab90a00640 1 -- 172.21.15.49:0/111310438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c0755c0 con 0x7fab8c15fd90 2024-04-15T15:19:38.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.750+0000 7fab83400640 1 -- 172.21.15.49:0/111310438 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab84005d10 con 0x7fab8c15fd90 2024-04-15T15:19:38.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.751+0000 7fab83400640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fab6c06c8f0 0x7fab6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:38.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.751+0000 7fab83400640 1 -- 172.21.15.49:0/111310438 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fab8408c810 con 0x7fab8c15fd90 2024-04-15T15:19:38.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.751+0000 7fab8b400640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fab6c06c8f0 0x7fab6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:38.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.752+0000 7fab8b400640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fab6c06c8f0 0x7fab6c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab7c036000 tx=0x7fab7c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:38.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.756+0000 7fab83400640 1 -- 172.21.15.49:0/111310438 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab84022030 con 0x7fab8c15fd90 2024-04-15T15:19:38.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.962+0000 7fab90a00640 1 -- 172.21.15.49:0/111310438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fab8c06dbe0 con 0x7fab8c15fd90 2024-04-15T15:19:38.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.963+0000 7fab83400640 1 -- 172.21.15.49:0/111310438 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fab84010040 con 0x7fab8c15fd90 2024-04-15T15:19:38.965 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:38.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.965+0000 7fab90a00640 1 -- 172.21.15.49:0/111310438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fab6c06c8f0 msgr2=0x7fab6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.965+0000 7fab90a00640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fab6c06c8f0 0x7fab6c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab7c036000 tx=0x7fab7c03d040 comp rx=0 tx=0).stop 2024-04-15T15:19:38.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.965+0000 7fab90a00640 1 -- 172.21.15.49:0/111310438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15fd90 msgr2=0x7fab8c0b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:38.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.965+0000 7fab90a00640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15fd90 0x7fab8c0b8b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fab8400efc0 tx=0x7fab8400c490 comp rx=0 tx=0).stop 2024-04-15T15:19:38.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.966+0000 7fab90a00640 1 -- 172.21.15.49:0/111310438 shutdown_connections 2024-04-15T15:19:38.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.966+0000 7fab90a00640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fab6c06c8f0 0x7fab6c06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.966+0000 7fab90a00640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab8c15fd90 0x7fab8c0b8b50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.966+0000 7fab90a00640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab8c15c680 0x7fab8c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.966+0000 7fab90a00640 1 --2- 172.21.15.49:0/111310438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab8c0b6c70 0x7fab8c1f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:38.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.966+0000 7fab90a00640 1 -- 172.21.15.49:0/111310438 >> 172.21.15.49:0/111310438 conn(0x7fab8c073480 msgr2=0x7fab8c160930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:38.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.966+0000 7fab90a00640 1 -- 172.21.15.49:0/111310438 shutdown_connections 2024-04-15T15:19:38.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:38.966+0000 7fab90a00640 1 -- 172.21.15.49:0/111310438 wait complete. 2024-04-15T15:19:38.990 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:39.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.229+0000 7f54ddc00640 1 Processor -- start 2024-04-15T15:19:39.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.230+0000 7f54ddc00640 1 -- start start 2024-04-15T15:19:39.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.231+0000 7f54ddc00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f54d8147c90 0x7f54d806f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.231+0000 7f54ddc00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f54d806f750 0x7f54d81626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.231+0000 7f54ddc00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d80aebd0 con 0x7f54d81478b0 2024-04-15T15:19:39.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.231+0000 7f54ddc00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f54d806fbb0 con 0x7f54d8147c90 2024-04-15T15:19:39.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.231+0000 7f54ddc00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f54d80aee50 con 0x7f54d806f750 2024-04-15T15:19:39.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.232+0000 7f54d7400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f54d8147c90 0x7f54d806f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.232+0000 7f54d7400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f54d8147c90 0x7f54d806f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38362/0 (socket says 172.21.15.49:38362) 2024-04-15T15:19:39.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.232+0000 7f54d7400640 1 -- 172.21.15.49:0/1897427472 learned_addr learned my addr 172.21.15.49:0/1897427472 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:39.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.232+0000 7f54dd200640 1 --2- 172.21.15.49:0/1897427472 >> v2:172.21.15.107:3300/0 conn(0x7f54d806f750 0x7f54d81626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.232+0000 7f54d7400640 1 -- 172.21.15.49:0/1897427472 >> v2:172.21.15.107:3300/0 conn(0x7f54d806f750 msgr2=0x7f54d81626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:39.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.232+0000 7f54d7400640 1 --2- 172.21.15.49:0/1897427472 >> v2:172.21.15.107:3300/0 conn(0x7f54d806f750 0x7f54d81626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.233+0000 7f54d7400640 1 -- 172.21.15.49:0/1897427472 >> v1:172.21.15.107:6789/0 conn(0x7f54d81478b0 legacy=0x7f54d80afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:39.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.233+0000 7f54d7400640 1 -- 172.21.15.49:0/1897427472 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d80af110 con 0x7f54d8147c90 2024-04-15T15:19:39.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.233+0000 7f54dd200640 1 --2- 172.21.15.49:0/1897427472 >> v2:172.21.15.107:3300/0 conn(0x7f54d806f750 0x7f54d81626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:39.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.234+0000 7f54d7400640 1 --2- 172.21.15.49:0/1897427472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d8147c90 0x7f54d806f210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f54d806c860 tx=0x7f54c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f8958d4664b29e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:39.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.234+0000 7f54d6a00640 1 -- 172.21.15.49:0/1897427472 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4035020 con 0x7f54d8147c90 2024-04-15T15:19:39.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.234+0000 7f54d6a00640 1 -- 172.21.15.49:0/1897427472 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c4031870 con 0x7f54d8147c90 2024-04-15T15:19:39.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.234+0000 7f54d6a00640 1 -- 172.21.15.49:0/1897427472 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4031c40 con 0x7f54d8147c90 2024-04-15T15:19:39.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.235+0000 7f54ddc00640 1 -- 172.21.15.49:0/1897427472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d8147c90 msgr2=0x7f54d806f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.235+0000 7f54ddc00640 1 --2- 172.21.15.49:0/1897427472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d8147c90 0x7f54d806f210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f54d806c860 tx=0x7f54c402b910 comp rx=0 tx=0).stop 2024-04-15T15:19:39.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.235+0000 7f54ddc00640 1 -- 172.21.15.49:0/1897427472 shutdown_connections 2024-04-15T15:19:39.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.235+0000 7f54ddc00640 1 --2- 172.21.15.49:0/1897427472 >> v2:172.21.15.107:3300/0 conn(0x7f54d806f750 0x7f54d81626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.235+0000 7f54ddc00640 1 --2- 172.21.15.49:0/1897427472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d8147c90 0x7f54d806f210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.235+0000 7f54ddc00640 1 -- 172.21.15.49:0/1897427472 >> 172.21.15.49:0/1897427472 conn(0x7f54d806b930 msgr2=0x7f54d806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:39.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.235+0000 7f54ddc00640 1 -- 172.21.15.49:0/1897427472 shutdown_connections 2024-04-15T15:19:39.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.235+0000 7f54ddc00640 1 -- 172.21.15.49:0/1897427472 wait complete. 2024-04-15T15:19:39.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.237+0000 7f54ddc00640 1 Processor -- start 2024-04-15T15:19:39.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.237+0000 7f54ddc00640 1 -- start start 2024-04-15T15:19:39.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.238+0000 7f54ddc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d806f750 0x7f54d80ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.238+0000 7f54ddc00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54d81478b0 0x7f54d8136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.238+0000 7f54ddc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54d8147c90 0x7f54d81611c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.238+0000 7f54ddc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8162c00 con 0x7f54d806f750 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.238+0000 7f54ddc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54d80af6d0 con 0x7f54d8147c90 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.238+0000 7f54ddc00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f54d8162d80 con 0x7f54d81478b0 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.238+0000 7f54d7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d806f750 0x7f54d80ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.238+0000 7f54d7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d806f750 0x7f54d80ced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38368/0 (socket says 172.21.15.49:38368) 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54d7e00640 1 -- 172.21.15.49:0/4270603547 learned_addr learned my addr 172.21.15.49:0/4270603547 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54dd200640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54d8147c90 0x7f54d81611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54d7400640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54d81478b0 0x7f54d8136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54d7e00640 1 -- 172.21.15.49:0/4270603547 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54d81478b0 msgr2=0x7f54d8136ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54d7e00640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54d81478b0 0x7f54d8136ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54d7e00640 1 -- 172.21.15.49:0/4270603547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54d8147c90 msgr2=0x7f54d81611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54d7e00640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54d8147c90 0x7f54d81611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54d7e00640 1 -- 172.21.15.49:0/4270603547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d80cce40 con 0x7f54d806f750 2024-04-15T15:19:39.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54dd200640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54d8147c90 0x7f54d81611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:39.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54d7e00640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d806f750 0x7f54d80ced00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54c800c430 tx=0x7f54c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:39.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.239+0000 7f54cfe00640 1 -- 172.21.15.49:0/4270603547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8011790 con 0x7f54d806f750 2024-04-15T15:19:39.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.240+0000 7f54cfe00640 1 -- 172.21.15.49:0/4270603547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c8017070 con 0x7f54d806f750 2024-04-15T15:19:39.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.240+0000 7f54cfe00640 1 -- 172.21.15.49:0/4270603547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8011bb0 con 0x7f54d806f750 2024-04-15T15:19:39.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.240+0000 7f54ddc00640 1 -- 172.21.15.49:0/4270603547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d80cd120 con 0x7f54d806f750 2024-04-15T15:19:39.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.240+0000 7f54ddc00640 1 -- 172.21.15.49:0/4270603547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d80cd6e0 con 0x7f54d806f750 2024-04-15T15:19:39.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.241+0000 7f54ddc00640 1 -- 172.21.15.49:0/4270603547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d806c680 con 0x7f54d806f750 2024-04-15T15:19:39.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.242+0000 7f54cfe00640 1 -- 172.21.15.49:0/4270603547 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54c801c460 con 0x7f54d806f750 2024-04-15T15:19:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.244+0000 7f54cfe00640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54b806c920 0x7f54b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.244+0000 7f54cfe00640 1 -- 172.21.15.49:0/4270603547 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f54c808baf0 con 0x7f54d806f750 2024-04-15T15:19:39.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.244+0000 7f54d7400640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54b806c920 0x7f54b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.245+0000 7f54d7400640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54b806c920 0x7f54b806ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f54c4036000 tx=0x7f54c4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:39.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.249+0000 7f54cfe00640 1 -- 172.21.15.49:0/4270603547 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54d806c680 con 0x7f54d806f750 2024-04-15T15:19:39.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.453+0000 7f54ddc00640 1 -- 172.21.15.49:0/4270603547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f54d8142d50 con 0x7f54d806f750 2024-04-15T15:19:39.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.454+0000 7f54cfe00640 1 -- 172.21.15.49:0/4270603547 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f54c8059e00 con 0x7f54d806f750 2024-04-15T15:19:39.456 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:39.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 -- 172.21.15.49:0/4270603547 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54b806c920 msgr2=0x7f54b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54b806c920 0x7f54b806ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f54c4036000 tx=0x7f54c4008030 comp rx=0 tx=0).stop 2024-04-15T15:19:39.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 -- 172.21.15.49:0/4270603547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d806f750 msgr2=0x7f54d80ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d806f750 0x7f54d80ced00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54c800c430 tx=0x7f54c800c8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 -- 172.21.15.49:0/4270603547 shutdown_connections 2024-04-15T15:19:39.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54b806c920 0x7f54b806ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54d81478b0 0x7f54d8136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54d8147c90 0x7f54d81611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 --2- 172.21.15.49:0/4270603547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54d806f750 0x7f54d80ced00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 -- 172.21.15.49:0/4270603547 >> 172.21.15.49:0/4270603547 conn(0x7f54d806b930 msgr2=0x7f54d81421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:39.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 -- 172.21.15.49:0/4270603547 shutdown_connections 2024-04-15T15:19:39.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.457+0000 7f54ddc00640 1 -- 172.21.15.49:0/4270603547 wait complete. 2024-04-15T15:19:39.481 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:19:39.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.706+0000 7f2021400640 1 Processor -- start 2024-04-15T15:19:39.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.707+0000 7f2021400640 1 -- start start 2024-04-15T15:19:39.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.708+0000 7f2021400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f201c0d1550 0x7f201c0d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.708+0000 7f2021400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f201c0d32e0 0x7f201c0d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.708+0000 7f2021400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f201c0b78f0 con 0x7f201c15eab0 2024-04-15T15:19:39.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.708+0000 7f2021400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f201c0d1ef0 con 0x7f201c0d1550 2024-04-15T15:19:39.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.708+0000 7f2021400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f201c0d2e50 con 0x7f201c0d32e0 2024-04-15T15:19:39.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.709+0000 7f201b400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f201c0d1550 0x7f201c0d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.709+0000 7f201b400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f201c0d1550 0x7f201c0d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38370/0 (socket says 172.21.15.49:38370) 2024-04-15T15:19:39.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.709+0000 7f201b400640 1 -- 172.21.15.49:0/313584064 learned_addr learned my addr 172.21.15.49:0/313584064 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:39.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.709+0000 7f2020a00640 1 --2- 172.21.15.49:0/313584064 >> v2:172.21.15.107:3300/0 conn(0x7f201c0d32e0 0x7f201c0d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.709+0000 7f201b400640 1 -- 172.21.15.49:0/313584064 >> v2:172.21.15.107:3300/0 conn(0x7f201c0d32e0 msgr2=0x7f201c0d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:39.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.709+0000 7f201b400640 1 --2- 172.21.15.49:0/313584064 >> v2:172.21.15.107:3300/0 conn(0x7f201c0d32e0 0x7f201c0d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.709+0000 7f201b400640 1 -- 172.21.15.49:0/313584064 >> v1:172.21.15.107:6789/0 conn(0x7f201c15eab0 legacy=0x7f201c162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:39.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.709+0000 7f201b400640 1 -- 172.21.15.49:0/313584064 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c162d40 con 0x7f201c0d1550 2024-04-15T15:19:39.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.711+0000 7f201b400640 1 --2- 172.21.15.49:0/313584064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d1550 0x7f201c0d19b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f201c08d710 tx=0x7f200402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6691c382bd2ffcad server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:39.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.711+0000 7f201aa00640 1 -- 172.21.15.49:0/313584064 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004035020 con 0x7f201c0d1550 2024-04-15T15:19:39.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.711+0000 7f201aa00640 1 -- 172.21.15.49:0/313584064 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2004031870 con 0x7f201c0d1550 2024-04-15T15:19:39.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.711+0000 7f201aa00640 1 -- 172.21.15.49:0/313584064 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004031c40 con 0x7f201c0d1550 2024-04-15T15:19:39.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.711+0000 7f2021400640 1 -- 172.21.15.49:0/313584064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d1550 msgr2=0x7f201c0d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.711+0000 7f2021400640 1 --2- 172.21.15.49:0/313584064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d1550 0x7f201c0d19b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f201c08d710 tx=0x7f200402b910 comp rx=0 tx=0).stop 2024-04-15T15:19:39.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.712+0000 7f2021400640 1 -- 172.21.15.49:0/313584064 shutdown_connections 2024-04-15T15:19:39.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.712+0000 7f2021400640 1 --2- 172.21.15.49:0/313584064 >> v2:172.21.15.107:3300/0 conn(0x7f201c0d32e0 0x7f201c0d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.712+0000 7f2021400640 1 --2- 172.21.15.49:0/313584064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d1550 0x7f201c0d19b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.712+0000 7f2021400640 1 -- 172.21.15.49:0/313584064 >> 172.21.15.49:0/313584064 conn(0x7f201c073240 msgr2=0x7f201c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:39.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.712+0000 7f2021400640 1 -- 172.21.15.49:0/313584064 shutdown_connections 2024-04-15T15:19:39.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.712+0000 7f2021400640 1 -- 172.21.15.49:0/313584064 wait complete. 2024-04-15T15:19:39.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.713+0000 7f2021400640 1 Processor -- start 2024-04-15T15:19:39.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.714+0000 7f2021400640 1 -- start start 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.714+0000 7f2021400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0d1550 0x7f201c14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.714+0000 7f2021400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d32e0 0x7f201c15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f2021400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f201c15eab0 0x7f201c158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f2021400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f201c0d2ae0 con 0x7f201c0d32e0 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f2021400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f201c1632f0 con 0x7f201c0d1550 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f2021400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f201c15ee90 con 0x7f201c15eab0 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f201b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d32e0 0x7f201c15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f201b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d32e0 0x7f201c15a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38382/0 (socket says 172.21.15.49:38382) 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f201b400640 1 -- 172.21.15.49:0/290926933 learned_addr learned my addr 172.21.15.49:0/290926933 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f201be00640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0d1550 0x7f201c14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f201b400640 1 -- 172.21.15.49:0/290926933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f201c15eab0 msgr2=0x7f201c158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f201b400640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f201c15eab0 0x7f201c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f201b400640 1 -- 172.21.15.49:0/290926933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0d1550 msgr2=0x7f201c14daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f201b400640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0d1550 0x7f201c14daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f201b400640 1 -- 172.21.15.49:0/290926933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c0cc5f0 con 0x7f201c0d32e0 2024-04-15T15:19:39.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.715+0000 7f201be00640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0d1550 0x7f201c14daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.716+0000 7f201b400640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d32e0 0x7f201c15a900 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2004009960 tx=0x7f20040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.716+0000 7f2013e00640 1 -- 172.21.15.49:0/290926933 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004040070 con 0x7f201c0d32e0 2024-04-15T15:19:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.716+0000 7f2013e00640 1 -- 172.21.15.49:0/290926933 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2004030d00 con 0x7f201c0d32e0 2024-04-15T15:19:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.716+0000 7f2021400640 1 -- 172.21.15.49:0/290926933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201c15b330 con 0x7f201c0d32e0 2024-04-15T15:19:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.716+0000 7f2013e00640 1 -- 172.21.15.49:0/290926933 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004035040 con 0x7f201c0d32e0 2024-04-15T15:19:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.716+0000 7f2021400640 1 -- 172.21.15.49:0/290926933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201c15b8f0 con 0x7f201c0d32e0 2024-04-15T15:19:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.717+0000 7f2021400640 1 -- 172.21.15.49:0/290926933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201c0755c0 con 0x7f201c0d32e0 2024-04-15T15:19:39.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.718+0000 7f2013e00640 1 -- 172.21.15.49:0/290926933 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2004038020 con 0x7f201c0d32e0 2024-04-15T15:19:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.719+0000 7f2013e00640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff806c8f0 0x7f1ff806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.719+0000 7f2013e00640 1 -- 172.21.15.49:0/290926933 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f20040ada00 con 0x7f201c0d32e0 2024-04-15T15:19:39.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.720+0000 7f201be00640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff806c8f0 0x7f1ff806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.720+0000 7f201be00640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff806c8f0 0x7f1ff806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f200c00a830 tx=0x7f200c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:39.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.724+0000 7f2013e00640 1 -- 172.21.15.49:0/290926933 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f201c0b78f0 con 0x7f201c0d32e0 2024-04-15T15:19:39.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.944+0000 7f2021400640 1 -- 172.21.15.49:0/290926933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f201c0d2460 con 0x7f201c0d32e0 2024-04-15T15:19:39.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.945+0000 7f2013e00640 1 -- 172.21.15.49:0/290926933 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f200407bda0 con 0x7f201c0d32e0 2024-04-15T15:19:39.947 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2024-04-15T15:19:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 -- 172.21.15.49:0/290926933 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff806c8f0 msgr2=0x7f1ff806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff806c8f0 0x7f1ff806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f200c00a830 tx=0x7f200c008040 comp rx=0 tx=0).stop 2024-04-15T15:19:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 -- 172.21.15.49:0/290926933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d32e0 msgr2=0x7f201c15a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d32e0 0x7f201c15a900 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2004009960 tx=0x7f20040046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 -- 172.21.15.49:0/290926933 shutdown_connections 2024-04-15T15:19:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1ff806c8f0 0x7f1ff806edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f201c15eab0 0x7f201c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201c0d32e0 0x7f201c15a900 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 --2- 172.21.15.49:0/290926933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0d1550 0x7f201c14daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 -- 172.21.15.49:0/290926933 >> 172.21.15.49:0/290926933 conn(0x7f201c073240 msgr2=0x7f201c159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 -- 172.21.15.49:0/290926933 shutdown_connections 2024-04-15T15:19:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:39.948+0000 7f2021400640 1 -- 172.21.15.49:0/290926933 wait complete. 2024-04-15T15:19:39.969 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-15T15:19:39.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:39.969 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-15T15:19:40.069 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:19:40.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:40.069 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-15T15:19:40.120 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:19:40.120 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:40.178 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:19:40.178 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:40.227 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:19:40.228 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:19:40.277 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.513+0000 7f1f7b600640 1 Processor -- start 2024-04-15T15:19:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.514+0000 7f1f7b600640 1 -- start start 2024-04-15T15:19:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.515+0000 7f1f7b600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1f7406cab0 0x7f1f7415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.515+0000 7f1f7b600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7406ce90 0x7f1f740ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.515+0000 7f1f7b600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f740cdb00 con 0x7f1f740ce7e0 2024-04-15T15:19:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.515+0000 7f1f7b600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f1f7415d790 con 0x7f1f7406cab0 2024-04-15T15:19:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.515+0000 7f1f7b600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7415da70 con 0x7f1f7406ce90 2024-04-15T15:19:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.516+0000 7f1f78c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1f7406cab0 0x7f1f7415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.516+0000 7f1f73e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7406ce90 0x7f1f740ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.516+0000 7f1f79600640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f1f740ce7e0 0x7f1f7406b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:43734/0 (socket says 172.21.15.49:43734) 2024-04-15T15:19:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.516+0000 7f1f79600640 1 -- 172.21.15.49:0/3599366302 learned_addr learned my addr 172.21.15.49:0/3599366302 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.516+0000 7f1f78c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1f7406cab0 0x7f1f7415ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:46266/0 (socket says 172.21.15.49:46266) 2024-04-15T15:19:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.516+0000 7f1f73e00640 1 --2- 172.21.15.49:0/3599366302 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7406ce90 0x7f1f740ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:34770/0 (socket says 172.21.15.49:34770) 2024-04-15T15:19:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.517+0000 7f1f73e00640 1 -- 172.21.15.49:0/3599366302 >> v2:172.21.15.107:3300/0 conn(0x7f1f7406cab0 msgr2=0x7f1f7415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.517+0000 7f1f73e00640 1 --2- 172.21.15.49:0/3599366302 >> v2:172.21.15.107:3300/0 conn(0x7f1f7406cab0 0x7f1f7415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.517+0000 7f1f73e00640 1 -- 172.21.15.49:0/3599366302 >> v1:172.21.15.107:6789/0 conn(0x7f1f740ce7e0 legacy=0x7f1f7406b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.517+0000 7f1f73e00640 1 -- 172.21.15.49:0/3599366302 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7415dd50 con 0x7f1f7406ce90 2024-04-15T15:19:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.517+0000 7f1f78c00640 1 --2- 172.21.15.49:0/3599366302 >> v2:172.21.15.107:3300/0 conn(0x7f1f7406cab0 0x7f1f7415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:19:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.518+0000 7f1f73e00640 1 --2- 172.21.15.49:0/3599366302 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7406ce90 0x7f1f740ce2a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009d90 tx=0x7f1f5c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8216fd6f611beb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:40.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.519+0000 7f1f73400640 1 -- 172.21.15.49:0/3599366302 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c039070 con 0x7f1f7406ce90 2024-04-15T15:19:40.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.519+0000 7f1f73400640 1 -- 172.21.15.49:0/3599366302 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c02cc50 con 0x7f1f7406ce90 2024-04-15T15:19:40.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.519+0000 7f1f73400640 1 -- 172.21.15.49:0/3599366302 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c034440 con 0x7f1f7406ce90 2024-04-15T15:19:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.519+0000 7f1f7b600640 1 -- 172.21.15.49:0/3599366302 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7406ce90 msgr2=0x7f1f740ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.519+0000 7f1f7b600640 1 --2- 172.21.15.49:0/3599366302 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7406ce90 0x7f1f740ce2a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009d90 tx=0x7f1f5c02bea0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.520+0000 7f1f7b600640 1 -- 172.21.15.49:0/3599366302 shutdown_connections 2024-04-15T15:19:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.520+0000 7f1f7b600640 1 --2- 172.21.15.49:0/3599366302 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7406ce90 0x7f1f740ce2a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.520+0000 7f1f7b600640 1 --2- 172.21.15.49:0/3599366302 >> v2:172.21.15.107:3300/0 conn(0x7f1f7406cab0 0x7f1f7415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.520+0000 7f1f7b600640 1 -- 172.21.15.49:0/3599366302 >> 172.21.15.49:0/3599366302 conn(0x7f1f7406f860 msgr2=0x7f1f74072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.520+0000 7f1f7b600640 1 -- 172.21.15.49:0/3599366302 shutdown_connections 2024-04-15T15:19:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.520+0000 7f1f7b600640 1 -- 172.21.15.49:0/3599366302 wait complete. 2024-04-15T15:19:40.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.521+0000 7f1f7b600640 1 Processor -- start 2024-04-15T15:19:40.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.522+0000 7f1f7b600640 1 -- start start 2024-04-15T15:19:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f7b600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7406cab0 0x7f1f74156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f7b600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f7406ce90 0x7f1f74155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f7b600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f740ce7e0 0x7f1f7415df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f7b600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7406c040 con 0x7f1f7406ce90 2024-04-15T15:19:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f7b600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7415d410 con 0x7f1f7406cab0 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f7b600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7406c1c0 con 0x7f1f740ce7e0 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f73e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f7406ce90 0x7f1f74155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f73e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f7406ce90 0x7f1f74155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38386/0 (socket says 172.21.15.49:38386) 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f73e00640 1 -- 172.21.15.49:0/2118654627 learned_addr learned my addr 172.21.15.49:0/2118654627 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f78c00640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7406cab0 0x7f1f74156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.523+0000 7f1f79600640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f740ce7e0 0x7f1f7415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f73e00640 1 -- 172.21.15.49:0/2118654627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f740ce7e0 msgr2=0x7f1f7415df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f73e00640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f740ce7e0 0x7f1f7415df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f73e00640 1 -- 172.21.15.49:0/2118654627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7406cab0 msgr2=0x7f1f74156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f73e00640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7406cab0 0x7f1f74156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f73e00640 1 -- 172.21.15.49:0/2118654627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f74155d30 con 0x7f1f7406ce90 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f73e00640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f7406ce90 0x7f1f74155400 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009d90 tx=0x7f1f5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f70c00640 1 -- 172.21.15.49:0/2118654627 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c039050 con 0x7f1f7406ce90 2024-04-15T15:19:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f70c00640 1 -- 172.21.15.49:0/2118654627 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c045070 con 0x7f1f7406ce90 2024-04-15T15:19:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f70c00640 1 -- 172.21.15.49:0/2118654627 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c03e610 con 0x7f1f7406ce90 2024-04-15T15:19:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f7b600640 1 -- 172.21.15.49:0/2118654627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f74156010 con 0x7f1f7406ce90 2024-04-15T15:19:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.524+0000 7f1f7b600640 1 -- 172.21.15.49:0/2118654627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f741565d0 con 0x7f1f7406ce90 2024-04-15T15:19:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.526+0000 7f1f7b600640 1 -- 172.21.15.49:0/2118654627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f74068c80 con 0x7f1f7406ce90 2024-04-15T15:19:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.527+0000 7f1f70c00640 1 -- 172.21.15.49:0/2118654627 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f5c03e7b0 con 0x7f1f7406ce90 2024-04-15T15:19:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.527+0000 7f1f70c00640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f5006c8f0 0x7f1f5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.528+0000 7f1f70c00640 1 -- 172.21.15.49:0/2118654627 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f1f5c0ae790 con 0x7f1f7406ce90 2024-04-15T15:19:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.528+0000 7f1f78c00640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f5006c8f0 0x7f1f5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.528+0000 7f1f78c00640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f5006c8f0 0x7f1f5006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009560 tx=0x7f1f64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:40.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.533+0000 7f1f70c00640 1 -- 172.21.15.49:0/2118654627 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f5c07cb30 con 0x7f1f7406ce90 2024-04-15T15:19:40.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.740+0000 7f1f7b600640 1 -- 172.21.15.49:0/2118654627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f7415d070 con 0x7f1f7406ce90 2024-04-15T15:19:40.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.742+0000 7f1f70c00640 1 -- 172.21.15.49:0/2118654627 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16854 (secure 0 0 0) 0x7f1f5c07c430 con 0x7f1f7406ce90 2024-04-15T15:19:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:40.745 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:19:32.083681+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:40.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 -- 172.21.15.49:0/2118654627 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f5006c8f0 msgr2=0x7f1f5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:40.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f5006c8f0 0x7f1f5006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009560 tx=0x7f1f64008040 comp rx=0 tx=0).stop 2024-04-15T15:19:40.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 -- 172.21.15.49:0/2118654627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f7406ce90 msgr2=0x7f1f74155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:40.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f7406ce90 0x7f1f74155400 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009d90 tx=0x7f1f5c004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:40.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 -- 172.21.15.49:0/2118654627 shutdown_connections 2024-04-15T15:19:40.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f5006c8f0 0x7f1f5006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f740ce7e0 0x7f1f7415df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f7406ce90 0x7f1f74155400 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 --2- 172.21.15.49:0/2118654627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7406cab0 0x7f1f74156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 -- 172.21.15.49:0/2118654627 >> 172.21.15.49:0/2118654627 conn(0x7f1f7406f860 msgr2=0x7f1f740b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:40.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.745+0000 7f1f7b600640 1 -- 172.21.15.49:0/2118654627 shutdown_connections 2024-04-15T15:19:40.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.746+0000 7f1f7b600640 1 -- 172.21.15.49:0/2118654627 wait complete. 2024-04-15T15:19:40.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:40.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:40.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.984+0000 7f17f7200640 1 Processor -- start 2024-04-15T15:19:40.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.984+0000 7f17f7200640 1 -- start start 2024-04-15T15:19:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.985+0000 7f17f7200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f17f80d26b0 0x7f17f80b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.985+0000 7f17f7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 0x7f17f80d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.985+0000 7f17f7200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17f80cfef0 con 0x7f17f8161f80 2024-04-15T15:19:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.985+0000 7f17f7200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f17f80d2bb0 con 0x7f17f80d26b0 2024-04-15T15:19:40.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.985+0000 7f17f7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f17f80d2e90 con 0x7f17f80b7e80 2024-04-15T15:19:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.986+0000 7f17f6800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 0x7f17f80d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.986+0000 7f17f5e00640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f17f8161f80 0x7f17f80cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:43744/0 (socket says 172.21.15.49:43744) 2024-04-15T15:19:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.986+0000 7f17f5e00640 1 -- 172.21.15.49:0/2079761325 learned_addr learned my addr 172.21.15.49:0/2079761325 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.986+0000 7f17f6800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 0x7f17f80d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:34790/0 (socket says 172.21.15.49:34790) 2024-04-15T15:19:40.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.986+0000 7f17f5400640 1 --2- 172.21.15.49:0/2079761325 >> v2:172.21.15.107:3300/0 conn(0x7f17f80d26b0 0x7f17f80b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.987+0000 7f17f6800640 1 -- 172.21.15.49:0/2079761325 >> v2:172.21.15.107:3300/0 conn(0x7f17f80d26b0 msgr2=0x7f17f80b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:40.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.987+0000 7f17f6800640 1 --2- 172.21.15.49:0/2079761325 >> v2:172.21.15.107:3300/0 conn(0x7f17f80d26b0 0x7f17f80b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.987+0000 7f17f6800640 1 -- 172.21.15.49:0/2079761325 >> v1:172.21.15.107:6789/0 conn(0x7f17f8161f80 legacy=0x7f17f80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:40.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.987+0000 7f17f4a00640 1 -- 172.21.15.49:0/2079761325 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209529873 0 0) 0x7f17f80cfef0 con 0x7f17f8161f80 2024-04-15T15:19:40.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.987+0000 7f17f6800640 1 -- 172.21.15.49:0/2079761325 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f80d3170 con 0x7f17f80b7e80 2024-04-15T15:19:40.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.988+0000 7f17f6800640 1 --2- 172.21.15.49:0/2079761325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 0x7f17f80d1580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f17f00097a0 tx=0x7f17f002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f2706193992674 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:40.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.989+0000 7f17f4a00640 1 -- 172.21.15.49:0/2079761325 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0035020 con 0x7f17f80b7e80 2024-04-15T15:19:40.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.989+0000 7f17f4a00640 1 -- 172.21.15.49:0/2079761325 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f0031810 con 0x7f17f80b7e80 2024-04-15T15:19:40.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.989+0000 7f17f4a00640 1 -- 172.21.15.49:0/2079761325 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0031be0 con 0x7f17f80b7e80 2024-04-15T15:19:40.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.989+0000 7f17f7200640 1 -- 172.21.15.49:0/2079761325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 msgr2=0x7f17f80d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:40.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.989+0000 7f17f7200640 1 --2- 172.21.15.49:0/2079761325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 0x7f17f80d1580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f17f00097a0 tx=0x7f17f002b8b0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.990+0000 7f17f7200640 1 -- 172.21.15.49:0/2079761325 shutdown_connections 2024-04-15T15:19:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.990+0000 7f17f7200640 1 --2- 172.21.15.49:0/2079761325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 0x7f17f80d1580 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.990+0000 7f17f7200640 1 --2- 172.21.15.49:0/2079761325 >> v2:172.21.15.107:3300/0 conn(0x7f17f80d26b0 0x7f17f80b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.990+0000 7f17f7200640 1 -- 172.21.15.49:0/2079761325 >> 172.21.15.49:0/2079761325 conn(0x7f17f806ce40 msgr2=0x7f17f806d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.990+0000 7f17f7200640 1 -- 172.21.15.49:0/2079761325 shutdown_connections 2024-04-15T15:19:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.990+0000 7f17f7200640 1 -- 172.21.15.49:0/2079761325 wait complete. 2024-04-15T15:19:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.992+0000 7f17f7200640 1 Processor -- start 2024-04-15T15:19:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.992+0000 7f17f7200640 1 -- start start 2024-04-15T15:19:40.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.993+0000 7f17f7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 0x7f17f815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.993+0000 7f17f7200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17f80d26b0 0x7f17f8155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.993+0000 7f17f7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f8161f80 0x7f17f81f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.993+0000 7f17f7200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17f80d1ac0 con 0x7f17f80d26b0 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.993+0000 7f17f7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17f8162360 con 0x7f17f8161f80 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.993+0000 7f17f7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f17f80d1c40 con 0x7f17f80b7e80 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.993+0000 7f17f5400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17f80d26b0 0x7f17f8155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.993+0000 7f17f5400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17f80d26b0 0x7f17f8155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38394/0 (socket says 172.21.15.49:38394) 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.993+0000 7f17f5400640 1 -- 172.21.15.49:0/453596438 learned_addr learned my addr 172.21.15.49:0/453596438 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.993+0000 7f17f5e00640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 0x7f17f815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.994+0000 7f17f6800640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f8161f80 0x7f17f81f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:40.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.994+0000 7f17f5400640 1 -- 172.21.15.49:0/453596438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 msgr2=0x7f17f815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.994+0000 7f17f5400640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 0x7f17f815a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.994+0000 7f17f5400640 1 -- 172.21.15.49:0/453596438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f8161f80 msgr2=0x7f17f81f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.994+0000 7f17f5400640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f8161f80 0x7f17f81f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.994+0000 7f17f5400640 1 -- 172.21.15.49:0/453596438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f8155750 con 0x7f17f80d26b0 2024-04-15T15:19:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.994+0000 7f17f6800640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f8161f80 0x7f17f81f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.994+0000 7f17f5400640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17f80d26b0 0x7f17f8155210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f17e000e980 tx=0x7f17e000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:40.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.994+0000 7f17ee000640 1 -- 172.21.15.49:0/453596438 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0017070 con 0x7f17f80d26b0 2024-04-15T15:19:40.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.995+0000 7f17ee000640 1 -- 172.21.15.49:0/453596438 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e000ca00 con 0x7f17f80d26b0 2024-04-15T15:19:40.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.995+0000 7f17ee000640 1 -- 172.21.15.49:0/453596438 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e000cdd0 con 0x7f17f80d26b0 2024-04-15T15:19:40.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.995+0000 7f17f7200640 1 -- 172.21.15.49:0/453596438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f815fd60 con 0x7f17f80d26b0 2024-04-15T15:19:40.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.995+0000 7f17f7200640 1 -- 172.21.15.49:0/453596438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f8155cb0 con 0x7f17f80d26b0 2024-04-15T15:19:40.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.996+0000 7f17f7200640 1 -- 172.21.15.49:0/453596438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f80755c0 con 0x7f17f80d26b0 2024-04-15T15:19:40.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.997+0000 7f17ee000640 1 -- 172.21.15.49:0/453596438 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17e001a070 con 0x7f17f80d26b0 2024-04-15T15:19:40.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.998+0000 7f17ee000640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17d406c8f0 0x7f17d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:40.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.998+0000 7f17ee000640 1 -- 172.21.15.49:0/453596438 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f17e008bc70 con 0x7f17f80d26b0 2024-04-15T15:19:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.998+0000 7f17f5e00640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17d406c8f0 0x7f17d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:40.999+0000 7f17f5e00640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17d406c8f0 0x7f17d406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f17e80012e0 tx=0x7f17e8005bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:41.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.003+0000 7f17ee000640 1 -- 172.21.15.49:0/453596438 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f17f8155750 con 0x7f17f80d26b0 2024-04-15T15:19:41.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.209+0000 7f17f7200640 1 -- 172.21.15.49:0/453596438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f17f80d2490 con 0x7f17f80d26b0 2024-04-15T15:19:41.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.210+0000 7f17ee000640 1 -- 172.21.15.49:0/453596438 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f17e0059f80 con 0x7f17f80d26b0 2024-04-15T15:19:41.212 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:41.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.212+0000 7f17f7200640 1 -- 172.21.15.49:0/453596438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17d406c8f0 msgr2=0x7f17d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.212+0000 7f17f7200640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17d406c8f0 0x7f17d406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f17e80012e0 tx=0x7f17e8005bf0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.212+0000 7f17f7200640 1 -- 172.21.15.49:0/453596438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17f80d26b0 msgr2=0x7f17f8155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.212+0000 7f17f7200640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17f80d26b0 0x7f17f8155210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f17e000e980 tx=0x7f17e000ee40 comp rx=0 tx=0).stop 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.212+0000 7f17f7200640 1 -- 172.21.15.49:0/453596438 shutdown_connections 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.212+0000 7f17f7200640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17d406c8f0 0x7f17d406edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.212+0000 7f17f7200640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f8161f80 0x7f17f81f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.213+0000 7f17f7200640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17f80d26b0 0x7f17f8155210 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.213+0000 7f17f7200640 1 --2- 172.21.15.49:0/453596438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17f80b7e80 0x7f17f815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.213+0000 7f17f7200640 1 -- 172.21.15.49:0/453596438 >> 172.21.15.49:0/453596438 conn(0x7f17f806ce40 msgr2=0x7f17f8153870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.213+0000 7f17f7200640 1 -- 172.21.15.49:0/453596438 shutdown_connections 2024-04-15T15:19:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.213+0000 7f17f7200640 1 -- 172.21.15.49:0/453596438 wait complete. 2024-04-15T15:19:41.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:41.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.452+0000 7f89ee800640 1 Processor -- start 2024-04-15T15:19:41.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.453+0000 7f89ee800640 1 -- start start 2024-04-15T15:19:41.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.454+0000 7f89ee800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f89e8147c90 0x7f89e806f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.454+0000 7f89ee800640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89e80aebd0 con 0x7f89e81478b0 2024-04-15T15:19:41.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.454+0000 7f89ee800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89e80aee50 con 0x7f89e806f750 2024-04-15T15:19:41.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.454+0000 7f89ee800640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f89e806fbb0 con 0x7f89e8147c90 2024-04-15T15:19:41.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.455+0000 7f89ed400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f89e81478b0 0x7f89e80afbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37674/0 (socket says 172.21.15.49:37674) 2024-04-15T15:19:41.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.455+0000 7f89ed400640 1 -- 172.21.15.49:0/2792539242 learned_addr learned my addr 172.21.15.49:0/2792539242 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:41.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.455+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3736710231 0 0) 0x7f89dc001110 con 0x7f89e81478b0 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.455+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89d0003520 con 0x7f89e81478b0 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.455+0000 7f89eca00640 1 --2- 172.21.15.49:0/2792539242 >> v2:172.21.15.107:3300/0 conn(0x7f89e8147c90 0x7f89e806f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.455+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 401135858 0 0) 0x7f89d0003520 con 0x7f89e81478b0 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.455+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 >> v2:172.21.15.107:3300/0 conn(0x7f89e8147c90 msgr2=0x7f89e806f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.455+0000 7f89e7e00640 1 --2- 172.21.15.49:0/2792539242 >> v2:172.21.15.107:3300/0 conn(0x7f89e8147c90 0x7f89e806f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.455+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 >> v1:172.21.15.107:6789/0 conn(0x7f89e806f750 legacy=0x7f89e81626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.455+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e80af110 con 0x7f89e81478b0 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f89dc001840 con 0x7f89e81478b0 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 >> v1:172.21.15.49:6789/0 conn(0x7f89e81478b0 legacy=0x7f89e80afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89e7e00640 1 --2- 172.21.15.49:0/2792539242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89d0003a00 0x7f89d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89e7e00640 1 --2- 172.21.15.49:0/2792539242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0004800 0x7f89d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89e7e00640 1 --2- 172.21.15.49:0/2792539242 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89d00052e0 0x7f89d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f89d000c200 con 0x7f89d0003a00 2024-04-15T15:19:41.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89d000c3b0 con 0x7f89d0004800 2024-04-15T15:19:41.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f89d000c5b0 con 0x7f89d00052e0 2024-04-15T15:19:41.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89eca00640 1 --2- 172.21.15.49:0/2792539242 >> v2:172.21.15.107:3300/0 conn(0x7f89e8147c90 0x7f89e806f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:41.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89ed400640 1 --2- 172.21.15.49:0/2792539242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89d0003a00 0x7f89d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89ed400640 1 -- 172.21.15.49:0/2792539242 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89d00052e0 msgr2=0x7f89d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:41.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89ed400640 1 --2- 172.21.15.49:0/2792539242 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89d00052e0 0x7f89d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89ed400640 1 -- 172.21.15.49:0/2792539242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0004800 msgr2=0x7f89d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:41.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89ed400640 1 --2- 172.21.15.49:0/2792539242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0004800 0x7f89d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89ed400640 1 -- 172.21.15.49:0/2792539242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d0003520 con 0x7f89d0003a00 2024-04-15T15:19:41.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.456+0000 7f89ede00640 1 -- 172.21.15.49:0/2792539242 reap_dead start 2024-04-15T15:19:41.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.457+0000 7f89ed400640 1 --2- 172.21.15.49:0/2792539242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89d0003a00 0x7f89d0003e00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89dc007fd0 tx=0x7f89d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568633dbaf55f0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:41.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.458+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc007e30 con 0x7f89d0003a00 2024-04-15T15:19:41.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.458+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89dc030070 con 0x7f89d0003a00 2024-04-15T15:19:41.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.458+0000 7f89e7e00640 1 -- 172.21.15.49:0/2792539242 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc02ce30 con 0x7f89d0003a00 2024-04-15T15:19:41.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.458+0000 7f89ee800640 1 -- 172.21.15.49:0/2792539242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89d0003a00 msgr2=0x7f89d0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.458+0000 7f89ee800640 1 --2- 172.21.15.49:0/2792539242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89d0003a00 0x7f89d0003e00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89dc007fd0 tx=0x7f89d0005830 comp rx=0 tx=0).stop 2024-04-15T15:19:41.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.458+0000 7f89ee800640 1 -- 172.21.15.49:0/2792539242 shutdown_connections 2024-04-15T15:19:41.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.458+0000 7f89ee800640 1 --2- 172.21.15.49:0/2792539242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89d0003a00 0x7f89d0003e00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.458+0000 7f89ee800640 1 -- 172.21.15.49:0/2792539242 >> 172.21.15.49:0/2792539242 conn(0x7f89e806b930 msgr2=0x7f89e806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:41.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.458+0000 7f89ee800640 1 -- 172.21.15.49:0/2792539242 shutdown_connections 2024-04-15T15:19:41.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.458+0000 7f89ee800640 1 -- 172.21.15.49:0/2792539242 wait complete. 2024-04-15T15:19:41.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.460+0000 7f89ee800640 1 Processor -- start 2024-04-15T15:19:41.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.460+0000 7f89ee800640 1 -- start start 2024-04-15T15:19:41.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.461+0000 7f89ee800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0003a00 0x7f89e81407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.461+0000 7f89ee800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89e815ed60 0x7f89e8142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.461+0000 7f89ee800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e815d1a0 0x7f89e815d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.461+0000 7f89ee800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f89e806f210 con 0x7f89e815ed60 2024-04-15T15:19:41.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.461+0000 7f89ee800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89e80af6d0 con 0x7f89d0003a00 2024-04-15T15:19:41.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.461+0000 7f89ee800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f89e806f390 con 0x7f89e815d1a0 2024-04-15T15:19:41.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89eca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89e815ed60 0x7f89e8142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89eca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89e815ed60 0x7f89e8142280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38410/0 (socket says 172.21.15.49:38410) 2024-04-15T15:19:41.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89eca00640 1 -- 172.21.15.49:0/772243786 learned_addr learned my addr 172.21.15.49:0/772243786 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:41.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89ed400640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0003a00 0x7f89e81407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89ede00640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e815d1a0 0x7f89e815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89eca00640 1 -- 172.21.15.49:0/772243786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e815d1a0 msgr2=0x7f89e815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89eca00640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e815d1a0 0x7f89e815d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89eca00640 1 -- 172.21.15.49:0/772243786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0003a00 msgr2=0x7f89e81407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89eca00640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0003a00 0x7f89e81407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89eca00640 1 -- 172.21.15.49:0/772243786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e8141dd0 con 0x7f89e815ed60 2024-04-15T15:19:41.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.462+0000 7f89ede00640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e815d1a0 0x7f89e815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:41.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.463+0000 7f89eca00640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89e815ed60 0x7f89e8142280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c820 tx=0x7f89d800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:41.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.463+0000 7f89e5600640 1 -- 172.21.15.49:0/772243786 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80079d0 con 0x7f89e815ed60 2024-04-15T15:19:41.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.463+0000 7f89e5600640 1 -- 172.21.15.49:0/772243786 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d8007b70 con 0x7f89e815ed60 2024-04-15T15:19:41.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.463+0000 7f89ee800640 1 -- 172.21.15.49:0/772243786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e815db60 con 0x7f89e815ed60 2024-04-15T15:19:41.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.463+0000 7f89e5600640 1 -- 172.21.15.49:0/772243786 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80113f0 con 0x7f89e815ed60 2024-04-15T15:19:41.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.463+0000 7f89ee800640 1 -- 172.21.15.49:0/772243786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e813fcd0 con 0x7f89e815ed60 2024-04-15T15:19:41.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.464+0000 7f89ee800640 1 -- 172.21.15.49:0/772243786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e80755c0 con 0x7f89e815ed60 2024-04-15T15:19:41.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.465+0000 7f89e5600640 1 -- 172.21.15.49:0/772243786 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89d8011590 con 0x7f89e815ed60 2024-04-15T15:19:41.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.466+0000 7f89e5600640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f89cc06c8f0 0x7f89cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.466+0000 7f89e5600640 1 -- 172.21.15.49:0/772243786 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f89d808bdb0 con 0x7f89e815ed60 2024-04-15T15:19:41.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.466+0000 7f89ed400640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f89cc06c8f0 0x7f89cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.467+0000 7f89ed400640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f89cc06c8f0 0x7f89cc06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89dc034000 tx=0x7f89dc009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:41.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.470+0000 7f89e5600640 1 -- 172.21.15.49:0/772243786 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89d8090070 con 0x7f89e815ed60 2024-04-15T15:19:41.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.666+0000 7f89ee800640 1 -- 172.21.15.49:0/772243786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f89e806ea00 con 0x7f89e815ed60 2024-04-15T15:19:41.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.667+0000 7f89e5600640 1 -- 172.21.15.49:0/772243786 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f89d8015020 con 0x7f89e815ed60 2024-04-15T15:19:41.668 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:41.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.669+0000 7f89ee800640 1 -- 172.21.15.49:0/772243786 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f89cc06c8f0 msgr2=0x7f89cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.669+0000 7f89ee800640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f89cc06c8f0 0x7f89cc06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89dc034000 tx=0x7f89dc009030 comp rx=0 tx=0).stop 2024-04-15T15:19:41.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.669+0000 7f89ee800640 1 -- 172.21.15.49:0/772243786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89e815ed60 msgr2=0x7f89e8142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.669+0000 7f89ee800640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89e815ed60 0x7f89e8142280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c820 tx=0x7f89d800cce0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.669+0000 7f89ee800640 1 -- 172.21.15.49:0/772243786 shutdown_connections 2024-04-15T15:19:41.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.669+0000 7f89ee800640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f89cc06c8f0 0x7f89cc06edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.669+0000 7f89ee800640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e815d1a0 0x7f89e815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.669+0000 7f89ee800640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89e815ed60 0x7f89e8142280 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.669+0000 7f89ee800640 1 --2- 172.21.15.49:0/772243786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0003a00 0x7f89e81407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.669+0000 7f89ee800640 1 -- 172.21.15.49:0/772243786 >> 172.21.15.49:0/772243786 conn(0x7f89e806b930 msgr2=0x7f89e8144ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:41.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.670+0000 7f89ee800640 1 -- 172.21.15.49:0/772243786 shutdown_connections 2024-04-15T15:19:41.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.670+0000 7f89ee800640 1 -- 172.21.15.49:0/772243786 wait complete. 2024-04-15T15:19:41.689 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:41.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.917+0000 7f1b48200640 1 Processor -- start 2024-04-15T15:19:41.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.917+0000 7f1b48200640 1 -- start start 2024-04-15T15:19:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.919+0000 7f1b48200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1b400b5830 0x7f1b4015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.919+0000 7f1b48200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b40067a40 0x7f1b400b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.919+0000 7f1b48200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1b400b6150 0x7f1b40162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.919+0000 7f1b48200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f1b400b4e00 con 0x7f1b400b5830 2024-04-15T15:19:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.919+0000 7f1b48200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f1b400b41f0 con 0x7f1b400b6150 2024-04-15T15:19:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.919+0000 7f1b48200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1b400b44d0 con 0x7f1b40067a40 2024-04-15T15:19:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.919+0000 7f1b45800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1b400b5830 0x7f1b4015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.919+0000 7f1b45800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1b400b5830 0x7f1b4015ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38424/0 (socket says 172.21.15.49:38424) 2024-04-15T15:19:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.919+0000 7f1b45800640 1 -- 172.21.15.49:0/3090698649 learned_addr learned my addr 172.21.15.49:0/3090698649 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:41.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.920+0000 7f1b45800640 1 -- 172.21.15.49:0/3090698649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b40067a40 msgr2=0x7f1b400b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:41.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.920+0000 7f1b45800640 1 --2- 172.21.15.49:0/3090698649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b40067a40 0x7f1b400b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.920+0000 7f1b45800640 1 -- 172.21.15.49:0/3090698649 >> v2:172.21.15.107:3300/0 conn(0x7f1b400b6150 msgr2=0x7f1b40162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:41.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.920+0000 7f1b45800640 1 --2- 172.21.15.49:0/3090698649 >> v2:172.21.15.107:3300/0 conn(0x7f1b400b6150 0x7f1b40162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.920+0000 7f1b45800640 1 -- 172.21.15.49:0/3090698649 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b400b47b0 con 0x7f1b400b5830 2024-04-15T15:19:41.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.921+0000 7f1b45800640 1 --2- 172.21.15.49:0/3090698649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 0x7f1b4015ec80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1b400cd950 tx=0x7f1b3402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f5f4b789556d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.921+0000 7f1b3fe00640 1 -- 172.21.15.49:0/3090698649 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34035020 con 0x7f1b400b5830 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.921+0000 7f1b3fe00640 1 -- 172.21.15.49:0/3090698649 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3402d040 con 0x7f1b400b5830 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.921+0000 7f1b3fe00640 1 -- 172.21.15.49:0/3090698649 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34031a80 con 0x7f1b400b5830 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.922+0000 7f1b48200640 1 -- 172.21.15.49:0/3090698649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 msgr2=0x7f1b4015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.922+0000 7f1b48200640 1 --2- 172.21.15.49:0/3090698649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 0x7f1b4015ec80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1b400cd950 tx=0x7f1b3402b900 comp rx=0 tx=0).stop 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.922+0000 7f1b48200640 1 -- 172.21.15.49:0/3090698649 shutdown_connections 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.922+0000 7f1b48200640 1 --2- 172.21.15.49:0/3090698649 >> v2:172.21.15.107:3300/0 conn(0x7f1b400b6150 0x7f1b40162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.922+0000 7f1b48200640 1 --2- 172.21.15.49:0/3090698649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b40067a40 0x7f1b400b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.922+0000 7f1b48200640 1 --2- 172.21.15.49:0/3090698649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 0x7f1b4015ec80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.922+0000 7f1b48200640 1 -- 172.21.15.49:0/3090698649 >> 172.21.15.49:0/3090698649 conn(0x7f1b4006ea10 msgr2=0x7f1b4006ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.922+0000 7f1b48200640 1 -- 172.21.15.49:0/3090698649 shutdown_connections 2024-04-15T15:19:41.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.922+0000 7f1b48200640 1 -- 172.21.15.49:0/3090698649 wait complete. 2024-04-15T15:19:41.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.924+0000 7f1b48200640 1 Processor -- start 2024-04-15T15:19:41.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.924+0000 7f1b48200640 1 -- start start 2024-04-15T15:19:41.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.925+0000 7f1b48200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b40067a40 0x7f1b4014c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.925+0000 7f1b48200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 0x7f1b401e8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.925+0000 7f1b48200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b400b6150 0x7f1b4015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.925+0000 7f1b48200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4015f340 con 0x7f1b400b5830 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.925+0000 7f1b48200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4015f1c0 con 0x7f1b400b6150 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.925+0000 7f1b48200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1b40162ad0 con 0x7f1b40067a40 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.925+0000 7f1b44e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 0x7f1b401e8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.925+0000 7f1b44e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 0x7f1b401e8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38434/0 (socket says 172.21.15.49:38434) 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.925+0000 7f1b44e00640 1 -- 172.21.15.49:0/2568737595 learned_addr learned my addr 172.21.15.49:0/2568737595 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.925+0000 7f1b45800640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b40067a40 0x7f1b4014c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b46200640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b400b6150 0x7f1b4015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b44e00640 1 -- 172.21.15.49:0/2568737595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b40067a40 msgr2=0x7f1b4014c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b44e00640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b40067a40 0x7f1b4014c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b44e00640 1 -- 172.21.15.49:0/2568737595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b400b6150 msgr2=0x7f1b4015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b44e00640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b400b6150 0x7f1b4015bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:41.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b44e00640 1 -- 172.21.15.49:0/2568737595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b401e93f0 con 0x7f1b400b5830 2024-04-15T15:19:41.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b46200640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b400b6150 0x7f1b4015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:41.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b44e00640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 0x7f1b401e8eb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b4a0 tx=0x7f1b2c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:41.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b3d600640 1 -- 172.21.15.49:0/2568737595 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c014070 con 0x7f1b400b5830 2024-04-15T15:19:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b3d600640 1 -- 172.21.15.49:0/2568737595 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b2c010610 con 0x7f1b400b5830 2024-04-15T15:19:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b3d600640 1 -- 172.21.15.49:0/2568737595 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c010a00 con 0x7f1b400b5830 2024-04-15T15:19:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.926+0000 7f1b48200640 1 -- 172.21.15.49:0/2568737595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b400ce3c0 con 0x7f1b400b5830 2024-04-15T15:19:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.927+0000 7f1b48200640 1 -- 172.21.15.49:0/2568737595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4015b490 con 0x7f1b400b5830 2024-04-15T15:19:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.928+0000 7f1b48200640 1 -- 172.21.15.49:0/2568737595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b400ce8e0 con 0x7f1b400b5830 2024-04-15T15:19:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.929+0000 7f1b3d600640 1 -- 172.21.15.49:0/2568737595 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b2c017070 con 0x7f1b400b5830 2024-04-15T15:19:41.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.929+0000 7f1b3d600640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1b2006c8f0 0x7f1b2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:41.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.930+0000 7f1b3d600640 1 -- 172.21.15.49:0/2568737595 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f1b2c08b590 con 0x7f1b400b5830 2024-04-15T15:19:41.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.930+0000 7f1b45800640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1b2006c8f0 0x7f1b2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:41.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.930+0000 7f1b45800640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1b2006c8f0 0x7f1b2006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1b34036000 tx=0x7f1b3403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:41.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:41.934+0000 7f1b3d600640 1 -- 172.21.15.49:0/2568737595 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b400ce8e0 con 0x7f1b400b5830 2024-04-15T15:19:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.136+0000 7f1b48200640 1 -- 172.21.15.49:0/2568737595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1b40068c80 con 0x7f1b400b5830 2024-04-15T15:19:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.137+0000 7f1b3d600640 1 -- 172.21.15.49:0/2568737595 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f1b401e93f0 con 0x7f1b400b5830 2024-04-15T15:19:42.139 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.139+0000 7f1b48200640 1 -- 172.21.15.49:0/2568737595 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1b2006c8f0 msgr2=0x7f1b2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.139+0000 7f1b48200640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1b2006c8f0 0x7f1b2006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1b34036000 tx=0x7f1b3403c040 comp rx=0 tx=0).stop 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.140+0000 7f1b48200640 1 -- 172.21.15.49:0/2568737595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 msgr2=0x7f1b401e8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.140+0000 7f1b48200640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 0x7f1b401e8eb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b4a0 tx=0x7f1b2c00b960 comp rx=0 tx=0).stop 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.140+0000 7f1b48200640 1 -- 172.21.15.49:0/2568737595 shutdown_connections 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.140+0000 7f1b48200640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1b2006c8f0 0x7f1b2006edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.140+0000 7f1b48200640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b400b6150 0x7f1b4015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.140+0000 7f1b48200640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b400b5830 0x7f1b401e8eb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.140+0000 7f1b48200640 1 --2- 172.21.15.49:0/2568737595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b40067a40 0x7f1b4014c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.140+0000 7f1b48200640 1 -- 172.21.15.49:0/2568737595 >> 172.21.15.49:0/2568737595 conn(0x7f1b4006ea10 msgr2=0x7f1b400aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.140+0000 7f1b48200640 1 -- 172.21.15.49:0/2568737595 shutdown_connections 2024-04-15T15:19:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.140+0000 7f1b48200640 1 -- 172.21.15.49:0/2568737595 wait complete. 2024-04-15T15:19:42.160 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:19:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.376+0000 7fe9cf200640 1 Processor -- start 2024-04-15T15:19:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.377+0000 7fe9cf200640 1 -- start start 2024-04-15T15:19:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.378+0000 7fe9cf200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d015d770 0x7fe9d00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.379+0000 7fe9cf200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d00cd940 con 0x7fe9d00b0cb0 2024-04-15T15:19:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.379+0000 7fe9cf200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d00cdc20 con 0x7fe9d00b59b0 2024-04-15T15:19:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.379+0000 7fe9cf200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d00ce550 con 0x7fe9d015d770 2024-04-15T15:19:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.379+0000 7fe9cde00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe9d00b0cb0 0x7fe9d00ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37684/0 (socket says 172.21.15.49:37684) 2024-04-15T15:19:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.379+0000 7fe9cde00640 1 -- 172.21.15.49:0/514417972 learned_addr learned my addr 172.21.15.49:0/514417972 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.379+0000 7fe9cd400640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d015d770 0x7fe9d00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.379+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099018746 0 0) 0x7fe9d00cd940 con 0x7fe9d00b0cb0 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.379+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9b4003520 con 0x7fe9d00b0cb0 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3579660112 0 0) 0x7fe9b4003520 con 0x7fe9d00b0cb0 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d015d770 msgr2=0x7fe9d00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d015d770 0x7fe9d00cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 >> v1:172.21.15.107:6789/0 conn(0x7fe9d00b59b0 legacy=0x7fe9d015dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d00cdf00 con 0x7fe9d00b0cb0 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fe9c0001670 con 0x7fe9d00b0cb0 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 >> v1:172.21.15.49:6789/0 conn(0x7fe9d00b0cb0 legacy=0x7fe9d00ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d015d770 0x7fe9b40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9b4004010 0x7fe9b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4004d40 0x7fe9b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d00ce550 con 0x7fe9b4004d40 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b400bb80 con 0x7fe9b4004010 2024-04-15T15:19:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b400bd80 con 0x7fe9d015d770 2024-04-15T15:19:42.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.380+0000 7fe9ce800640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4004d40 0x7fe9b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:42.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.381+0000 7fe9cde00640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d015d770 0x7fe9b40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:42.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.381+0000 7fe9cd400640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9b4004010 0x7fe9b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:42.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.381+0000 7fe9ce800640 1 -- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d015d770 msgr2=0x7fe9b40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:42.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.381+0000 7fe9ce800640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d015d770 0x7fe9b40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.381+0000 7fe9ce800640 1 -- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9b4004010 msgr2=0x7fe9b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:42.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.381+0000 7fe9ce800640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9b4004010 0x7fe9b4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.381+0000 7fe9ce800640 1 -- 172.21.15.49:0/514417972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b4003520 con 0x7fe9b4004d40 2024-04-15T15:19:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.382+0000 7fe9ce800640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4004d40 0x7fe9b40051c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8001e50 tx=0x7fe9b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41c929aaf4b3cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.382+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8001a70 con 0x7fe9b4004d40 2024-04-15T15:19:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.382+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c8003b70 con 0x7fe9b4004d40 2024-04-15T15:19:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.382+0000 7fe9cca00640 1 -- 172.21.15.49:0/514417972 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80050d0 con 0x7fe9b4004d40 2024-04-15T15:19:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.383+0000 7fe9cf200640 1 -- 172.21.15.49:0/514417972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4004d40 msgr2=0x7fe9b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.383+0000 7fe9cf200640 1 --2- 172.21.15.49:0/514417972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4004d40 0x7fe9b40051c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8001e50 tx=0x7fe9b4005830 comp rx=0 tx=0).stop 2024-04-15T15:19:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.383+0000 7fe9ce800640 1 -- 172.21.15.49:0/514417972 reap_dead start 2024-04-15T15:19:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.383+0000 7fe9cf200640 1 -- 172.21.15.49:0/514417972 shutdown_connections 2024-04-15T15:19:42.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.383+0000 7fe9cf200640 1 -- 172.21.15.49:0/514417972 >> 172.21.15.49:0/514417972 conn(0x7fe9d00b23a0 msgr2=0x7fe9d00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:42.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.383+0000 7fe9cf200640 1 -- 172.21.15.49:0/514417972 shutdown_connections 2024-04-15T15:19:42.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.383+0000 7fe9cf200640 1 -- 172.21.15.49:0/514417972 wait complete. 2024-04-15T15:19:42.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.385+0000 7fe9cf200640 1 Processor -- start 2024-04-15T15:19:42.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.385+0000 7fe9cf200640 1 -- start start 2024-04-15T15:19:42.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.386+0000 7fe9cf200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9b4004d40 0x7fe9d0158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:42.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.386+0000 7fe9cf200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d013f0e0 0x7fe9d013f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.386+0000 7fe9cf200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d0141030 0x7fe9d0155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.386+0000 7fe9cf200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d015e380 con 0x7fe9d013f0e0 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.386+0000 7fe9cf200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d00b1090 con 0x7fe9d0141030 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.386+0000 7fe9cf200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d015e500 con 0x7fe9b4004d40 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.386+0000 7fe9cd400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d013f0e0 0x7fe9d013f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.386+0000 7fe9cd400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d013f0e0 0x7fe9d013f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38444/0 (socket says 172.21.15.49:38444) 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.386+0000 7fe9cd400640 1 -- 172.21.15.49:0/1927368887 learned_addr learned my addr 172.21.15.49:0/1927368887 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.387+0000 7fe9cde00640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9b4004d40 0x7fe9d0158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.387+0000 7fe9ce800640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d0141030 0x7fe9d0155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.387+0000 7fe9cd400640 1 -- 172.21.15.49:0/1927368887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9b4004d40 msgr2=0x7fe9d0158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.387+0000 7fe9cd400640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9b4004d40 0x7fe9d0158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.387+0000 7fe9cd400640 1 -- 172.21.15.49:0/1927368887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d0141030 msgr2=0x7fe9d0155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.387+0000 7fe9cd400640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d0141030 0x7fe9d0155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.387+0000 7fe9cd400640 1 -- 172.21.15.49:0/1927368887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d013ff90 con 0x7fe9d013f0e0 2024-04-15T15:19:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.387+0000 7fe9cd400640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d013f0e0 0x7fe9d013f810 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800e7e0 tx=0x7fe9b800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.387+0000 7fe9c6000640 1 -- 172.21.15.49:0/1927368887 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8017070 con 0x7fe9d013f0e0 2024-04-15T15:19:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.388+0000 7fe9c6000640 1 -- 172.21.15.49:0/1927368887 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b800c6d0 con 0x7fe9d013f0e0 2024-04-15T15:19:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.388+0000 7fe9c6000640 1 -- 172.21.15.49:0/1927368887 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800f040 con 0x7fe9d013f0e0 2024-04-15T15:19:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.388+0000 7fe9cf200640 1 -- 172.21.15.49:0/1927368887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d0140270 con 0x7fe9d013f0e0 2024-04-15T15:19:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.388+0000 7fe9cf200640 1 -- 172.21.15.49:0/1927368887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d0140830 con 0x7fe9d013f0e0 2024-04-15T15:19:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.389+0000 7fe9cf200640 1 -- 172.21.15.49:0/1927368887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d00755c0 con 0x7fe9d013f0e0 2024-04-15T15:19:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.390+0000 7fe9c6000640 1 -- 172.21.15.49:0/1927368887 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9b801a070 con 0x7fe9d013f0e0 2024-04-15T15:19:42.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.391+0000 7fe9c6000640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe9ac06c8f0 0x7fe9ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:42.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.391+0000 7fe9c6000640 1 -- 172.21.15.49:0/1927368887 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fe9b808b8b0 con 0x7fe9d013f0e0 2024-04-15T15:19:42.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.391+0000 7fe9cde00640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe9ac06c8f0 0x7fe9ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:42.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.392+0000 7fe9cde00640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe9ac06c8f0 0x7fe9ac06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0001eb0 tx=0x7fe9c0001d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:42.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.396+0000 7fe9c6000640 1 -- 172.21.15.49:0/1927368887 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9b8059c50 con 0x7fe9d013f0e0 2024-04-15T15:19:42.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.615+0000 7fe9cf200640 1 -- 172.21.15.49:0/1927368887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe9d00b3040 con 0x7fe9d013f0e0 2024-04-15T15:19:42.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.616+0000 7fe9c6000640 1 -- 172.21.15.49:0/1927368887 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fe9b8055080 con 0x7fe9d013f0e0 2024-04-15T15:19:42.618 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2024-04-15T15:19:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.618+0000 7fe9cf200640 1 -- 172.21.15.49:0/1927368887 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe9ac06c8f0 msgr2=0x7fe9ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.618+0000 7fe9cf200640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe9ac06c8f0 0x7fe9ac06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0001eb0 tx=0x7fe9c0001d80 comp rx=0 tx=0).stop 2024-04-15T15:19:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.618+0000 7fe9cf200640 1 -- 172.21.15.49:0/1927368887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d013f0e0 msgr2=0x7fe9d013f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.618+0000 7fe9cf200640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d013f0e0 0x7fe9d013f810 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800e7e0 tx=0x7fe9b800eca0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.619+0000 7fe9cf200640 1 -- 172.21.15.49:0/1927368887 shutdown_connections 2024-04-15T15:19:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.619+0000 7fe9cf200640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe9ac06c8f0 0x7fe9ac06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.619+0000 7fe9cf200640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d0141030 0x7fe9d0155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.619+0000 7fe9cf200640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d013f0e0 0x7fe9d013f810 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.619+0000 7fe9cf200640 1 --2- 172.21.15.49:0/1927368887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9b4004d40 0x7fe9d0158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.619+0000 7fe9cf200640 1 -- 172.21.15.49:0/1927368887 >> 172.21.15.49:0/1927368887 conn(0x7fe9d00b23a0 msgr2=0x7fe9d00cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.619+0000 7fe9cf200640 1 -- 172.21.15.49:0/1927368887 shutdown_connections 2024-04-15T15:19:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:42.619+0000 7fe9cf200640 1 -- 172.21.15.49:0/1927368887 wait complete. 2024-04-15T15:19:42.642 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-15T15:19:42.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:42.642 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-15T15:19:42.744 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:19:42.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:19:42.745 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-15T15:19:42.794 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-15T15:19:42.794 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:43.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.033+0000 7fee69600640 1 Processor -- start 2024-04-15T15:19:43.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.033+0000 7fee69600640 1 -- start start 2024-04-15T15:19:43.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.035+0000 7fee69600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fee64162db0 0x7fee640b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.035+0000 7fee69600640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee6406f460 con 0x7fee640ce7f0 2024-04-15T15:19:43.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.035+0000 7fee69600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee640b4100 con 0x7fee640b0060 2024-04-15T15:19:43.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.035+0000 7fee69600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fee6406f790 con 0x7fee64162db0 2024-04-15T15:19:43.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.036+0000 7fee63400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fee640ce7f0 0x7fee640721e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37688/0 (socket says 172.21.15.49:37688) 2024-04-15T15:19:43.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.036+0000 7fee63400640 1 -- 172.21.15.49:0/3964149161 learned_addr learned my addr 172.21.15.49:0/3964149161 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:43.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.036+0000 7fee62a00640 1 --2- 172.21.15.49:0/3964149161 >> v2:172.21.15.107:3300/0 conn(0x7fee64162db0 0x7fee640b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.036+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054346465 0 0) 0x7fee6406f460 con 0x7fee640ce7f0 2024-04-15T15:19:43.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.036+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee48003520 con 0x7fee640ce7f0 2024-04-15T15:19:43.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.036+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1591735280 0 0) 0x7fee48003520 con 0x7fee640ce7f0 2024-04-15T15:19:43.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 >> v2:172.21.15.107:3300/0 conn(0x7fee64162db0 msgr2=0x7fee640b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:43.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 --2- 172.21.15.49:0/3964149161 >> v2:172.21.15.107:3300/0 conn(0x7fee64162db0 0x7fee640b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 >> v1:172.21.15.107:6789/0 conn(0x7fee640b0060 legacy=0x7fee6406fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee640b43e0 con 0x7fee640ce7f0 2024-04-15T15:19:43.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fee5c0019f0 con 0x7fee640ce7f0 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 >> v1:172.21.15.49:6789/0 conn(0x7fee640ce7f0 legacy=0x7fee640721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee48003a00 0x7fee48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee48004800 0x7fee48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee480051c0 0x7fee4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee6406f790 con 0x7fee480051c0 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee4800c0c0 con 0x7fee48003a00 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fee4800c2c0 con 0x7fee48004800 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee63400640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee480051c0 0x7fee4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.037+0000 7fee62a00640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee48003a00 0x7fee48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.038+0000 7fee62000640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee48004800 0x7fee48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.038+0000 7fee63400640 1 -- 172.21.15.49:0/3964149161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee48004800 msgr2=0x7fee48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.038+0000 7fee63400640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee48004800 0x7fee48004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.038+0000 7fee63400640 1 -- 172.21.15.49:0/3964149161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee48003a00 msgr2=0x7fee48003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:43.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.038+0000 7fee63400640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee48003a00 0x7fee48003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.038+0000 7fee63400640 1 -- 172.21.15.49:0/3964149161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee48003520 con 0x7fee480051c0 2024-04-15T15:19:43.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.038+0000 7fee62a00640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee48003a00 0x7fee48003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:43.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.039+0000 7fee63400640 1 -- 172.21.15.49:0/3964149161 reap_dead start 2024-04-15T15:19:43.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.039+0000 7fee63400640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee480051c0 0x7fee4800b9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00a790 tx=0x7fee48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c1fb6f4bbdcc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:43.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.039+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c00a3b0 con 0x7fee480051c0 2024-04-15T15:19:43.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.039+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5c034670 con 0x7fee480051c0 2024-04-15T15:19:43.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.039+0000 7fee61600640 1 -- 172.21.15.49:0/3964149161 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c034a60 con 0x7fee480051c0 2024-04-15T15:19:43.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.040+0000 7fee69600640 1 -- 172.21.15.49:0/3964149161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee480051c0 msgr2=0x7fee4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.040+0000 7fee69600640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee480051c0 0x7fee4800b9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00a790 tx=0x7fee48005830 comp rx=0 tx=0).stop 2024-04-15T15:19:43.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.040+0000 7fee69600640 1 -- 172.21.15.49:0/3964149161 shutdown_connections 2024-04-15T15:19:43.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.040+0000 7fee69600640 1 --2- 172.21.15.49:0/3964149161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee480051c0 0x7fee4800b9c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.040+0000 7fee69600640 1 -- 172.21.15.49:0/3964149161 >> 172.21.15.49:0/3964149161 conn(0x7fee640b27b0 msgr2=0x7fee640b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:43.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.040+0000 7fee69600640 1 -- 172.21.15.49:0/3964149161 shutdown_connections 2024-04-15T15:19:43.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.040+0000 7fee69600640 1 -- 172.21.15.49:0/3964149161 wait complete. 2024-04-15T15:19:43.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.042+0000 7fee69600640 1 Processor -- start 2024-04-15T15:19:43.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.042+0000 7fee69600640 1 -- start start 2024-04-15T15:19:43.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.043+0000 7fee69600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480051c0 0x7fee64164d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.043+0000 7fee69600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6414ccf0 0x7fee6415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.043+0000 7fee69600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6415c160 0x7fee64158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.043+0000 7fee69600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee64163310 con 0x7fee6415c160 2024-04-15T15:19:43.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.043+0000 7fee69600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee64163190 con 0x7fee480051c0 2024-04-15T15:19:43.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.043+0000 7fee69600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fee640cec80 con 0x7fee6414ccf0 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.043+0000 7fee63400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6415c160 0x7fee64158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.043+0000 7fee63400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6415c160 0x7fee64158820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38450/0 (socket says 172.21.15.49:38450) 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.043+0000 7fee63400640 1 -- 172.21.15.49:0/1631571502 learned_addr learned my addr 172.21.15.49:0/1631571502 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee62000640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6414ccf0 0x7fee6415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee63400640 1 -- 172.21.15.49:0/1631571502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6414ccf0 msgr2=0x7fee6415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee62a00640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480051c0 0x7fee64164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee63400640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6414ccf0 0x7fee6415a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee63400640 1 -- 172.21.15.49:0/1631571502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480051c0 msgr2=0x7fee64164d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee63400640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480051c0 0x7fee64164d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee63400640 1 -- 172.21.15.49:0/1631571502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee64156030 con 0x7fee6415c160 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee63400640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6415c160 0x7fee64158820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fee5c02cba0 tx=0x7fee640b45b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee5aa00640 1 -- 172.21.15.49:0/1631571502 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c043070 con 0x7fee6415c160 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee5aa00640 1 -- 172.21.15.49:0/1631571502 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5c001250 con 0x7fee6415c160 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee5aa00640 1 -- 172.21.15.49:0/1631571502 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c039040 con 0x7fee6415c160 2024-04-15T15:19:43.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.044+0000 7fee69600640 1 -- 172.21.15.49:0/1631571502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6415ae40 con 0x7fee6415c160 2024-04-15T15:19:43.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.045+0000 7fee69600640 1 -- 172.21.15.49:0/1631571502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6415b3a0 con 0x7fee6415c160 2024-04-15T15:19:43.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.046+0000 7fee69600640 1 -- 172.21.15.49:0/1631571502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee64068c80 con 0x7fee6415c160 2024-04-15T15:19:43.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.047+0000 7fee5aa00640 1 -- 172.21.15.49:0/1631571502 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee5c046070 con 0x7fee6415c160 2024-04-15T15:19:43.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.048+0000 7fee5aa00640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fee4406c920 0x7fee4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.048+0000 7fee5aa00640 1 -- 172.21.15.49:0/1631571502 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fee5c0acf00 con 0x7fee6415c160 2024-04-15T15:19:43.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.048+0000 7fee62a00640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fee4406c920 0x7fee4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.049+0000 7fee62a00640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fee4406c920 0x7fee4406ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee540099c0 tx=0x7fee54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:43.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.052+0000 7fee5aa00640 1 -- 172.21.15.49:0/1631571502 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee6415ae40 con 0x7fee6415c160 2024-04-15T15:19:43.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.299+0000 7fee69600640 1 -- 172.21.15.49:0/1631571502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee640b10a0 con 0x7fee6415c160 2024-04-15T15:19:43.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.300+0000 7fee5aa00640 1 -- 172.21.15.49:0/1631571502 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fee5c07b210 con 0x7fee6415c160 2024-04-15T15:19:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:43.302 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/203558139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":203558139},{"type":"v1","addr":"172.21.15.107:6833","nonce":203558139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3198352923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3198352923},{"type":"v1","addr":"172.21.15.49:6836","nonce":3198352923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1425337425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1425337425},{"type":"v1","addr":"172.21.15.49:6837","nonce":1425337425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:18:45.325981+0000","modified":"2024-04-15T15:18:48.378286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2374398161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2374398161},{"type":"v1","addr":"172.21.15.107:6835","nonce":2374398161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-15T15:19:43.302 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-04-15T15:19:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 -- 172.21.15.49:0/1631571502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fee4406c920 msgr2=0x7fee4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fee4406c920 0x7fee4406ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee540099c0 tx=0x7fee54008040 comp rx=0 tx=0).stop 2024-04-15T15:19:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 -- 172.21.15.49:0/1631571502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6415c160 msgr2=0x7fee64158820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6415c160 0x7fee64158820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fee5c02cba0 tx=0x7fee640b45b0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 -- 172.21.15.49:0/1631571502 shutdown_connections 2024-04-15T15:19:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fee4406c920 0x7fee4406ede0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6414ccf0 0x7fee6415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6415c160 0x7fee64158820 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 --2- 172.21.15.49:0/1631571502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480051c0 0x7fee64164d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 -- 172.21.15.49:0/1631571502 >> 172.21.15.49:0/1631571502 conn(0x7fee640b27b0 msgr2=0x7fee640b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:43.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 -- 172.21.15.49:0/1631571502 shutdown_connections 2024-04-15T15:19:43.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.303+0000 7fee69600640 1 -- 172.21.15.49:0/1631571502 wait complete. 2024-04-15T15:19:43.326 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:43.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.544+0000 7f37aee00640 1 Processor -- start 2024-04-15T15:19:43.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.545+0000 7f37aee00640 1 -- start start 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.546+0000 7f37aee00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f37a80d1c90 0x7f37a8162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.546+0000 7f37aee00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a80cf110 0x7f37a80d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.547+0000 7f37aee00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37a80d0090 con 0x7f37a8162020 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.547+0000 7f37aee00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f37a8163440 con 0x7f37a80d1c90 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.547+0000 7f37aee00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f37a80b7b30 con 0x7f37a80cf110 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.547+0000 7f37ada00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f37a8162020 0x7f37a80d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37696/0 (socket says 172.21.15.49:37696) 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.547+0000 7f37ada00640 1 -- 172.21.15.49:0/2686704774 learned_addr learned my addr 172.21.15.49:0/2686704774 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.547+0000 7f37ae400640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a80cf110 0x7f37a80d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.547+0000 7f37ad000640 1 --2- 172.21.15.49:0/2686704774 >> v2:172.21.15.107:3300/0 conn(0x7f37a80d1c90 0x7f37a8162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.547+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860616271 0 0) 0x7f379c001110 con 0x7f37a8162020 2024-04-15T15:19:43.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.547+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3790003520 con 0x7f37a8162020 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4206037759 0 0) 0x7f3790003520 con 0x7f37a8162020 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a80cf110 msgr2=0x7f37a80d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a80cf110 0x7f37a80d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 >> v2:172.21.15.107:3300/0 conn(0x7f37a80d1c90 msgr2=0x7f37a8162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 --2- 172.21.15.49:0/2686704774 >> v2:172.21.15.107:3300/0 conn(0x7f37a80d1c90 0x7f37a8162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a80cfad0 con 0x7f37a8162020 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f379c001840 con 0x7f37a8162020 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 >> v1:172.21.15.49:6789/0 conn(0x7f37a8162020 legacy=0x7f37a80d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a80cf110 0x7f37900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3790003f30 0x7f3790004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3790004d40 0x7f37900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f37a8163440 con 0x7f3790004d40 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37a80b7b30 con 0x7f3790003f30 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f379000b9c0 con 0x7f37a80cf110 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.548+0000 7f37ae400640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3790004d40 0x7f37900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.549+0000 7f37ada00640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a80cf110 0x7f37900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.549+0000 7f37ad000640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3790003f30 0x7f3790004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.549+0000 7f37ae400640 1 -- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a80cf110 msgr2=0x7f37900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.549+0000 7f37ae400640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a80cf110 0x7f37900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.549+0000 7f37ae400640 1 -- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3790003f30 msgr2=0x7f3790004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.549+0000 7f37ae400640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3790003f30 0x7f3790004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.549+0000 7f37ae400640 1 -- 172.21.15.49:0/2686704774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3790003520 con 0x7f3790004d40 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.550+0000 7f37ae400640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3790004d40 0x7f37900051c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009f90 tx=0x7f3790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b12b42a259a93ae server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.550+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0039070 con 0x7f3790004d40 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.550+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a002cbc0 con 0x7f3790004d40 2024-04-15T15:19:43.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.550+0000 7f37a7e00640 1 -- 172.21.15.49:0/2686704774 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0009c10 con 0x7f3790004d40 2024-04-15T15:19:43.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.551+0000 7f37aee00640 1 -- 172.21.15.49:0/2686704774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3790004d40 msgr2=0x7f37900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.551+0000 7f37aee00640 1 --2- 172.21.15.49:0/2686704774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3790004d40 0x7f37900051c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009f90 tx=0x7f3790005830 comp rx=0 tx=0).stop 2024-04-15T15:19:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.551+0000 7f37ae400640 1 -- 172.21.15.49:0/2686704774 reap_dead start 2024-04-15T15:19:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.551+0000 7f37aee00640 1 -- 172.21.15.49:0/2686704774 shutdown_connections 2024-04-15T15:19:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.551+0000 7f37aee00640 1 -- 172.21.15.49:0/2686704774 >> 172.21.15.49:0/2686704774 conn(0x7f37a806cf00 msgr2=0x7f37a806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.551+0000 7f37aee00640 1 -- 172.21.15.49:0/2686704774 shutdown_connections 2024-04-15T15:19:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.551+0000 7f37aee00640 1 -- 172.21.15.49:0/2686704774 wait complete. 2024-04-15T15:19:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.553+0000 7f37aee00640 1 Processor -- start 2024-04-15T15:19:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.553+0000 7f37aee00640 1 -- start start 2024-04-15T15:19:43.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.554+0000 7f37aee00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a81f7a10 0x7f37a815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.554+0000 7f37aee00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37a81f8050 0x7f37a81f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.554+0000 7f37aee00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a815a9f0 0x7f37a815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.554+0000 7f37aee00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f37a80d2b90 con 0x7f37a81f8050 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.554+0000 7f37aee00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37a80cf880 con 0x7f37a81f7a10 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.554+0000 7f37aee00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f37a80d2e70 con 0x7f37a815a9f0 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.554+0000 7f37ad000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37a81f8050 0x7f37a81f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.554+0000 7f37ad000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37a81f8050 0x7f37a81f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38460/0 (socket says 172.21.15.49:38460) 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.554+0000 7f37ad000640 1 -- 172.21.15.49:0/240446614 learned_addr learned my addr 172.21.15.49:0/240446614 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37ada00640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a81f7a10 0x7f37a815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37ae400640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a815a9f0 0x7f37a815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37ad000640 1 -- 172.21.15.49:0/240446614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a815a9f0 msgr2=0x7f37a815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37ad000640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a815a9f0 0x7f37a815ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37ad000640 1 -- 172.21.15.49:0/240446614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a81f7a10 msgr2=0x7f37a815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37ad000640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a81f7a10 0x7f37a815ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37ad000640 1 -- 172.21.15.49:0/240446614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a81f7df0 con 0x7f37a81f8050 2024-04-15T15:19:43.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37ad000640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37a81f8050 0x7f37a81f84d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3798011630 tx=0x7f3798011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:43.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37a5600640 1 -- 172.21.15.49:0/240446614 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379800f920 con 0x7f37a81f8050 2024-04-15T15:19:43.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37a5600640 1 -- 172.21.15.49:0/240446614 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f379800fac0 con 0x7f37a81f8050 2024-04-15T15:19:43.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37a5600640 1 -- 172.21.15.49:0/240446614 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379800fe70 con 0x7f37a81f8050 2024-04-15T15:19:43.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.555+0000 7f37aee00640 1 -- 172.21.15.49:0/240446614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a815b3c0 con 0x7f37a81f8050 2024-04-15T15:19:43.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.556+0000 7f37aee00640 1 -- 172.21.15.49:0/240446614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a815b940 con 0x7f37a81f8050 2024-04-15T15:19:43.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.557+0000 7f37aee00640 1 -- 172.21.15.49:0/240446614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a80755c0 con 0x7f37a81f8050 2024-04-15T15:19:43.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.558+0000 7f37a5600640 1 -- 172.21.15.49:0/240446614 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3798015460 con 0x7f37a81f8050 2024-04-15T15:19:43.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.559+0000 7f37a5600640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f378c06c8f0 0x7f378c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:43.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.559+0000 7f37a5600640 1 -- 172.21.15.49:0/240446614 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f379808c210 con 0x7f37a81f8050 2024-04-15T15:19:43.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.559+0000 7f37ada00640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f378c06c8f0 0x7f378c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:43.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.560+0000 7f37ada00640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f378c06c8f0 0x7f378c06edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f37a814b430 tx=0x7f379c0047e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:43.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.563+0000 7f37a5600640 1 -- 172.21.15.49:0/240446614 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3798019030 con 0x7f37a81f8050 2024-04-15T15:19:43.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.768+0000 7f37aee00640 1 -- 172.21.15.49:0/240446614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37a80b7650 con 0x7f37a81f8050 2024-04-15T15:19:43.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.771+0000 7f37a5600640 1 -- 172.21.15.49:0/240446614 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16854 (secure 0 0 0) 0x7f3798059ed0 con 0x7f37a81f8050 2024-04-15T15:19:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:43.773 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:19:32.083681+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:43.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 -- 172.21.15.49:0/240446614 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f378c06c8f0 msgr2=0x7f378c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f378c06c8f0 0x7f378c06edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f37a814b430 tx=0x7f379c0047e0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 -- 172.21.15.49:0/240446614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37a81f8050 msgr2=0x7f37a81f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:43.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37a81f8050 0x7f37a81f84d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3798011630 tx=0x7f3798011af0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 -- 172.21.15.49:0/240446614 shutdown_connections 2024-04-15T15:19:43.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f378c06c8f0 0x7f378c06edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a815a9f0 0x7f37a815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37a81f8050 0x7f37a81f84d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 --2- 172.21.15.49:0/240446614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a81f7a10 0x7f37a815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:43.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 -- 172.21.15.49:0/240446614 >> 172.21.15.49:0/240446614 conn(0x7f37a806cf00 msgr2=0x7f37a80cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:43.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 -- 172.21.15.49:0/240446614 shutdown_connections 2024-04-15T15:19:43.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:43.774+0000 7f37aee00640 1 -- 172.21.15.49:0/240446614 wait complete. 2024-04-15T15:19:43.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:18:43.124453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:18:44.307843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:43.797 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:44.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.033+0000 7f3b9fa00640 1 Processor -- start 2024-04-15T15:19:44.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.033+0000 7f3b9fa00640 1 -- start start 2024-04-15T15:19:44.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.035+0000 7f3b9fa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3b980ce650 0x7f3b9809ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.035+0000 7f3b9fa00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3b980703d0 0x7f3b9809b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.035+0000 7f3b9fa00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b9806d2e0 0x7f3b9806d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.035+0000 7f3b9fa00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9809bb40 con 0x7f3b980ce650 2024-04-15T15:19:44.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.035+0000 7f3b9fa00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9806ce50 con 0x7f3b980703d0 2024-04-15T15:19:44.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.035+0000 7f3b9fa00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f3b98071330 con 0x7f3b9806d2e0 2024-04-15T15:19:44.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.035+0000 7f3b9d000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3b980ce650 0x7f3b9809ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.035+0000 7f3b9d000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3b980ce650 0x7f3b9809ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56206/0 (socket says 172.21.15.49:56206) 2024-04-15T15:19:44.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.035+0000 7f3b9d000640 1 -- 172.21.15.49:0/2362783370 learned_addr learned my addr 172.21.15.49:0/2362783370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:44.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.036+0000 7f3b9da00640 1 --2- 172.21.15.49:0/2362783370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b9806d2e0 0x7f3b9806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.036+0000 7f3b97e00640 1 --2- 172.21.15.49:0/2362783370 >> v2:172.21.15.107:3300/0 conn(0x7f3b980703d0 0x7f3b9809b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.036+0000 7f3b9d000640 1 -- 172.21.15.49:0/2362783370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b9806d2e0 msgr2=0x7f3b9806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:44.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.036+0000 7f3b9d000640 1 --2- 172.21.15.49:0/2362783370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b9806d2e0 0x7f3b9806d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.036+0000 7f3b9d000640 1 -- 172.21.15.49:0/2362783370 >> v2:172.21.15.107:3300/0 conn(0x7f3b980703d0 msgr2=0x7f3b9809b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:44.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.036+0000 7f3b9d000640 1 --2- 172.21.15.49:0/2362783370 >> v2:172.21.15.107:3300/0 conn(0x7f3b980703d0 0x7f3b9809b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.036+0000 7f3b9d000640 1 -- 172.21.15.49:0/2362783370 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9806cae0 con 0x7f3b980ce650 2024-04-15T15:19:44.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.036+0000 7f3b97e00640 1 --2- 172.21.15.49:0/2362783370 >> v2:172.21.15.107:3300/0 conn(0x7f3b980703d0 0x7f3b9809b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:44.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.037+0000 7f3b9d000640 1 --2- 172.21.15.49:0/2362783370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b980ce650 0x7f3b9809ab20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3b98073710 tx=0x7f3b8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f61eec1ef7d0e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:44.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.037+0000 7f3b97400640 1 -- 172.21.15.49:0/2362783370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c035020 con 0x7f3b980ce650 2024-04-15T15:19:44.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.037+0000 7f3b97400640 1 -- 172.21.15.49:0/2362783370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b8c031870 con 0x7f3b980ce650 2024-04-15T15:19:44.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.037+0000 7f3b97400640 1 -- 172.21.15.49:0/2362783370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c031c40 con 0x7f3b980ce650 2024-04-15T15:19:44.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.038+0000 7f3b9fa00640 1 -- 172.21.15.49:0/2362783370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b980ce650 msgr2=0x7f3b9809ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.038+0000 7f3b9fa00640 1 --2- 172.21.15.49:0/2362783370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b980ce650 0x7f3b9809ab20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3b98073710 tx=0x7f3b8c02b910 comp rx=0 tx=0).stop 2024-04-15T15:19:44.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.038+0000 7f3b9fa00640 1 -- 172.21.15.49:0/2362783370 shutdown_connections 2024-04-15T15:19:44.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.038+0000 7f3b9fa00640 1 --2- 172.21.15.49:0/2362783370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b9806d2e0 0x7f3b9806d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.038+0000 7f3b9fa00640 1 --2- 172.21.15.49:0/2362783370 >> v2:172.21.15.107:3300/0 conn(0x7f3b980703d0 0x7f3b9809b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.038+0000 7f3b9fa00640 1 --2- 172.21.15.49:0/2362783370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b980ce650 0x7f3b9809ab20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.038+0000 7f3b9fa00640 1 -- 172.21.15.49:0/2362783370 >> 172.21.15.49:0/2362783370 conn(0x7f3b980d2720 msgr2=0x7f3b980d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:44.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.038+0000 7f3b9fa00640 1 -- 172.21.15.49:0/2362783370 shutdown_connections 2024-04-15T15:19:44.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.038+0000 7f3b9fa00640 1 -- 172.21.15.49:0/2362783370 wait complete. 2024-04-15T15:19:44.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.040+0000 7f3b9fa00640 1 Processor -- start 2024-04-15T15:19:44.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.040+0000 7f3b9fa00640 1 -- start start 2024-04-15T15:19:44.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.041+0000 7f3b9fa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b9806d2e0 0x7f3b98155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.041+0000 7f3b9fa00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b980703d0 0x7f3b980b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.041+0000 7f3b9fa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b980ce650 0x7f3b980b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.041+0000 7f3b9fa00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9809bd90 con 0x7f3b9806d2e0 2024-04-15T15:19:44.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.041+0000 7f3b9fa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3b98071610 con 0x7f3b980ce650 2024-04-15T15:19:44.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.041+0000 7f3b9fa00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f3b980cea30 con 0x7f3b980703d0 2024-04-15T15:19:44.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.041+0000 7f3b9d000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b9806d2e0 0x7f3b98155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.041+0000 7f3b9d000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b9806d2e0 0x7f3b98155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56212/0 (socket says 172.21.15.49:56212) 2024-04-15T15:19:44.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.041+0000 7f3b9d000640 1 -- 172.21.15.49:0/642313316 learned_addr learned my addr 172.21.15.49:0/642313316 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:44.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.042+0000 7f3b9d000640 1 -- 172.21.15.49:0/642313316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b980703d0 msgr2=0x7f3b980b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.042+0000 7f3b97e00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b980703d0 0x7f3b980b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.042+0000 7f3b9da00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b980ce650 0x7f3b980b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.042+0000 7f3b9d000640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b980703d0 0x7f3b980b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.042+0000 7f3b9d000640 1 -- 172.21.15.49:0/642313316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b980ce650 msgr2=0x7f3b980b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.042+0000 7f3b9d000640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b980ce650 0x7f3b980b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.042+0000 7f3b9d000640 1 -- 172.21.15.49:0/642313316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b980b8140 con 0x7f3b9806d2e0 2024-04-15T15:19:44.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.042+0000 7f3b9d000640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b9806d2e0 0x7f3b98155c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c02b950 tx=0x7f3b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:44.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.043+0000 7f3b94c00640 1 -- 172.21.15.49:0/642313316 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c040070 con 0x7f3b9806d2e0 2024-04-15T15:19:44.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.043+0000 7f3b94c00640 1 -- 172.21.15.49:0/642313316 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b8c030d00 con 0x7f3b9806d2e0 2024-04-15T15:19:44.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.043+0000 7f3b94c00640 1 -- 172.21.15.49:0/642313316 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c035040 con 0x7f3b9806d2e0 2024-04-15T15:19:44.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.043+0000 7f3b9fa00640 1 -- 172.21.15.49:0/642313316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b981f59b0 con 0x7f3b9806d2e0 2024-04-15T15:19:44.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.043+0000 7f3b9fa00640 1 -- 172.21.15.49:0/642313316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b980b7330 con 0x7f3b9806d2e0 2024-04-15T15:19:44.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.044+0000 7f3b9fa00640 1 -- 172.21.15.49:0/642313316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b98068c80 con 0x7f3b9806d2e0 2024-04-15T15:19:44.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.045+0000 7f3b94c00640 1 -- 172.21.15.49:0/642313316 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b8c04b060 con 0x7f3b9806d2e0 2024-04-15T15:19:44.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.046+0000 7f3b94c00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3b7806c8f0 0x7f3b7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.047+0000 7f3b94c00640 1 -- 172.21.15.49:0/642313316 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f3b8c0ad9f0 con 0x7f3b9806d2e0 2024-04-15T15:19:44.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.047+0000 7f3b97e00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3b7806c8f0 0x7f3b7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.047+0000 7f3b97e00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3b7806c8f0 0x7f3b7806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b981f7af0 tx=0x7f3b84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:44.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.051+0000 7f3b94c00640 1 -- 172.21.15.49:0/642313316 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b8c07bd90 con 0x7f3b9806d2e0 2024-04-15T15:19:44.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.263+0000 7f3b9fa00640 1 -- 172.21.15.49:0/642313316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b980ccf80 con 0x7f3b9806d2e0 2024-04-15T15:19:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.264+0000 7f3b94c00640 1 -- 172.21.15.49:0/642313316 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f3b8c07b690 con 0x7f3b9806d2e0 2024-04-15T15:19:44.266 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:44.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.267+0000 7f3b9fa00640 1 -- 172.21.15.49:0/642313316 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3b7806c8f0 msgr2=0x7f3b7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.267+0000 7f3b9fa00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3b7806c8f0 0x7f3b7806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b981f7af0 tx=0x7f3b84008040 comp rx=0 tx=0).stop 2024-04-15T15:19:44.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.267+0000 7f3b9fa00640 1 -- 172.21.15.49:0/642313316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b9806d2e0 msgr2=0x7f3b98155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.267+0000 7f3b9fa00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b9806d2e0 0x7f3b98155c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c02b950 tx=0x7f3b8c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.267+0000 7f3b9fa00640 1 -- 172.21.15.49:0/642313316 shutdown_connections 2024-04-15T15:19:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.267+0000 7f3b9fa00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3b7806c8f0 0x7f3b7806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.267+0000 7f3b9fa00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b980703d0 0x7f3b980b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.267+0000 7f3b9fa00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b980ce650 0x7f3b980b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.267+0000 7f3b9fa00640 1 --2- 172.21.15.49:0/642313316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b9806d2e0 0x7f3b98155c80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.267+0000 7f3b9fa00640 1 -- 172.21.15.49:0/642313316 >> 172.21.15.49:0/642313316 conn(0x7f3b980d2720 msgr2=0x7f3b980cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.268+0000 7f3b9fa00640 1 -- 172.21.15.49:0/642313316 shutdown_connections 2024-04-15T15:19:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.268+0000 7f3b9fa00640 1 -- 172.21.15.49:0/642313316 wait complete. 2024-04-15T15:19:44.291 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:19:44.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.517+0000 7f78bd600640 1 Processor -- start 2024-04-15T15:19:44.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.518+0000 7f78bd600640 1 -- start start 2024-04-15T15:19:44.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.519+0000 7f78bd600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f78b81461c0 0x7f78b8071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.519+0000 7f78bd600640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78b80cd2d0 con 0x7f78b8072420 2024-04-15T15:19:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.519+0000 7f78bd600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78b80cd5b0 con 0x7f78b80cebb0 2024-04-15T15:19:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.519+0000 7f78bd600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f78b80cf1d0 con 0x7f78b81461c0 2024-04-15T15:19:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.519+0000 7f78b7e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f78b81461c0 0x7f78b8071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.520+0000 7f78b7e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f78b81461c0 0x7f78b8071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56218/0 (socket says 172.21.15.49:56218) 2024-04-15T15:19:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.520+0000 7f78b7400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f78b8072420 0x7f78b80ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47900/0 (socket says 172.21.15.49:47900) 2024-04-15T15:19:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.520+0000 7f78b7e00640 1 -- 172.21.15.49:0/1065415943 learned_addr learned my addr 172.21.15.49:0/1065415943 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:44.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.520+0000 7f78b7e00640 1 -- 172.21.15.49:0/1065415943 >> v1:172.21.15.107:6789/0 conn(0x7f78b80cebb0 legacy=0x7f78b806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:44.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.520+0000 7f78b7e00640 1 -- 172.21.15.49:0/1065415943 >> v1:172.21.15.49:6789/0 conn(0x7f78b8072420 legacy=0x7f78b80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.520+0000 7f78b7e00640 1 -- 172.21.15.49:0/1065415943 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b80cd890 con 0x7f78b81461c0 2024-04-15T15:19:44.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.520+0000 7f78b6a00640 1 -- 172.21.15.49:0/1065415943 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740707683 0 0) 0x7f78b80cd2d0 con 0x7f78b8072420 2024-04-15T15:19:44.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.521+0000 7f78b7e00640 1 --2- 172.21.15.49:0/1065415943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b81461c0 0x7f78b8071ee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009870 tx=0x7f78a802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5295d7a10ca912d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.522+0000 7f78b6a00640 1 -- 172.21.15.49:0/1065415943 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8034070 con 0x7f78b81461c0 2024-04-15T15:19:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.522+0000 7f78b6a00640 1 -- 172.21.15.49:0/1065415943 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a8030860 con 0x7f78b81461c0 2024-04-15T15:19:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.522+0000 7f78b6a00640 1 -- 172.21.15.49:0/1065415943 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8030c30 con 0x7f78b81461c0 2024-04-15T15:19:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.522+0000 7f78bd600640 1 -- 172.21.15.49:0/1065415943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b81461c0 msgr2=0x7f78b8071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.522+0000 7f78bd600640 1 --2- 172.21.15.49:0/1065415943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b81461c0 0x7f78b8071ee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009870 tx=0x7f78a802b9a0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.522+0000 7f78bd600640 1 -- 172.21.15.49:0/1065415943 shutdown_connections 2024-04-15T15:19:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.522+0000 7f78bd600640 1 --2- 172.21.15.49:0/1065415943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b81461c0 0x7f78b8071ee0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.522+0000 7f78bd600640 1 -- 172.21.15.49:0/1065415943 >> 172.21.15.49:0/1065415943 conn(0x7f78b806b930 msgr2=0x7f78b806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:44.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.522+0000 7f78bd600640 1 -- 172.21.15.49:0/1065415943 shutdown_connections 2024-04-15T15:19:44.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.522+0000 7f78bd600640 1 -- 172.21.15.49:0/1065415943 wait complete. 2024-04-15T15:19:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.524+0000 7f78bd600640 1 Processor -- start 2024-04-15T15:19:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.524+0000 7f78bd600640 1 -- start start 2024-04-15T15:19:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.525+0000 7f78bd600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b8072420 0x7f78b8156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.525+0000 7f78bd600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b80cebb0 0x7f78b8158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.525+0000 7f78bd600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f78b81461c0 0x7f78b8159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.525+0000 7f78bd600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f78b80cd080 con 0x7f78b80cebb0 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.525+0000 7f78bd600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78b806e630 con 0x7f78b8072420 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.525+0000 7f78bd600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f78b806e7b0 con 0x7f78b81461c0 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.525+0000 7f78b7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b80cebb0 0x7f78b8158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78b7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b80cebb0 0x7f78b8158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56224/0 (socket says 172.21.15.49:56224) 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78b7400640 1 -- 172.21.15.49:0/4004975681 learned_addr learned my addr 172.21.15.49:0/4004975681 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78bcc00640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f78b81461c0 0x7f78b8159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78b7e00640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b8072420 0x7f78b8156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78b7400640 1 -- 172.21.15.49:0/4004975681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f78b81461c0 msgr2=0x7f78b8159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78b7400640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f78b81461c0 0x7f78b8159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78b7400640 1 -- 172.21.15.49:0/4004975681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b8072420 msgr2=0x7f78b8156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78b7400640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b8072420 0x7f78b8156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78b7400640 1 -- 172.21.15.49:0/4004975681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b815ac70 con 0x7f78b80cebb0 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78b7e00640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b8072420 0x7f78b8156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.526+0000 7f78b7400640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b80cebb0 0x7f78b8158c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c4d0 tx=0x7f78a400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.527+0000 7f78afe00640 1 -- 172.21.15.49:0/4004975681 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4018070 con 0x7f78b80cebb0 2024-04-15T15:19:44.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.527+0000 7f78afe00640 1 -- 172.21.15.49:0/4004975681 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a4007500 con 0x7f78b80cebb0 2024-04-15T15:19:44.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.527+0000 7f78afe00640 1 -- 172.21.15.49:0/4004975681 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4004a20 con 0x7f78b80cebb0 2024-04-15T15:19:44.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.527+0000 7f78bd600640 1 -- 172.21.15.49:0/4004975681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b813d5c0 con 0x7f78b80cebb0 2024-04-15T15:19:44.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.527+0000 7f78bd600640 1 -- 172.21.15.49:0/4004975681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b813db80 con 0x7f78b80cebb0 2024-04-15T15:19:44.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.528+0000 7f78bd600640 1 -- 172.21.15.49:0/4004975681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b815ae40 con 0x7f78b80cebb0 2024-04-15T15:19:44.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.529+0000 7f78afe00640 1 -- 172.21.15.49:0/4004975681 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78a4004bc0 con 0x7f78b80cebb0 2024-04-15T15:19:44.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.530+0000 7f78afe00640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f789806c8f0 0x7f789806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:44.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.530+0000 7f78afe00640 1 -- 172.21.15.49:0/4004975681 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f78a408aed0 con 0x7f78b80cebb0 2024-04-15T15:19:44.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.530+0000 7f78b7e00640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f789806c8f0 0x7f789806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:44.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.531+0000 7f78b7e00640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f789806c8f0 0x7f789806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78a8036000 tx=0x7f78a803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:44.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.535+0000 7f78afe00640 1 -- 172.21.15.49:0/4004975681 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f78a4059270 con 0x7f78b80cebb0 2024-04-15T15:19:44.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.740+0000 7f78bd600640 1 -- 172.21.15.49:0/4004975681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f78b80755c0 con 0x7f78b80cebb0 2024-04-15T15:19:44.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.741+0000 7f78afe00640 1 -- 172.21.15.49:0/4004975681 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f78a4058b70 con 0x7f78b80cebb0 2024-04-15T15:19:44.742 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:19:44.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.743+0000 7f78bd600640 1 -- 172.21.15.49:0/4004975681 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f789806c8f0 msgr2=0x7f789806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.743+0000 7f78bd600640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f789806c8f0 0x7f789806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78a8036000 tx=0x7f78a803b040 comp rx=0 tx=0).stop 2024-04-15T15:19:44.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.743+0000 7f78bd600640 1 -- 172.21.15.49:0/4004975681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b80cebb0 msgr2=0x7f78b8158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:44.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.743+0000 7f78bd600640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b80cebb0 0x7f78b8158c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c4d0 tx=0x7f78a400c990 comp rx=0 tx=0).stop 2024-04-15T15:19:44.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.743+0000 7f78bd600640 1 -- 172.21.15.49:0/4004975681 shutdown_connections 2024-04-15T15:19:44.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.743+0000 7f78bd600640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f789806c8f0 0x7f789806edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.743+0000 7f78bd600640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f78b81461c0 0x7f78b8159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.743+0000 7f78bd600640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b80cebb0 0x7f78b8158c10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.743+0000 7f78bd600640 1 --2- 172.21.15.49:0/4004975681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b8072420 0x7f78b8156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:44.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.743+0000 7f78bd600640 1 -- 172.21.15.49:0/4004975681 >> 172.21.15.49:0/4004975681 conn(0x7f78b806b930 msgr2=0x7f78b806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:44.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.744+0000 7f78bd600640 1 -- 172.21.15.49:0/4004975681 shutdown_connections 2024-04-15T15:19:44.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:44.744+0000 7f78bd600640 1 -- 172.21.15.49:0/4004975681 wait complete. 2024-04-15T15:19:44.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:19:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.000+0000 7faf4de00640 1 Processor -- start 2024-04-15T15:19:45.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.000+0000 7faf4de00640 1 -- start start 2024-04-15T15:19:45.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.002+0000 7faf4de00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faf481461c0 0x7faf48071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.002+0000 7faf4de00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faf480cebb0 0x7faf4806df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.002+0000 7faf4de00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf480cd2d0 con 0x7faf48072420 2024-04-15T15:19:45.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.002+0000 7faf4de00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7faf480cf1d0 con 0x7faf480cebb0 2024-04-15T15:19:45.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.002+0000 7faf4de00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7faf48072880 con 0x7faf481461c0 2024-04-15T15:19:45.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.002+0000 7faf4d400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faf480cebb0 0x7faf4806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.003+0000 7faf4d400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faf480cebb0 0x7faf4806df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56234/0 (socket says 172.21.15.49:56234) 2024-04-15T15:19:45.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.003+0000 7faf4d400640 1 -- 172.21.15.49:0/830576994 learned_addr learned my addr 172.21.15.49:0/830576994 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:45.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.003+0000 7faf4ca00640 1 --2- 172.21.15.49:0/830576994 >> v2:172.21.15.107:3300/0 conn(0x7faf481461c0 0x7faf48071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.003+0000 7faf4d400640 1 -- 172.21.15.49:0/830576994 >> v2:172.21.15.107:3300/0 conn(0x7faf481461c0 msgr2=0x7faf48071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:45.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.003+0000 7faf4d400640 1 --2- 172.21.15.49:0/830576994 >> v2:172.21.15.107:3300/0 conn(0x7faf481461c0 0x7faf48071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.003+0000 7faf4d400640 1 -- 172.21.15.49:0/830576994 >> v1:172.21.15.49:6789/0 conn(0x7faf48072420 legacy=0x7faf480ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.003+0000 7faf4d400640 1 -- 172.21.15.49:0/830576994 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf480cd5b0 con 0x7faf480cebb0 2024-04-15T15:19:45.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.004+0000 7faf4d400640 1 --2- 172.21.15.49:0/830576994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 0x7faf4806df20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faf480d05c0 tx=0x7faf4002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce94aaa7ca69d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.005+0000 7faf47400640 1 -- 172.21.15.49:0/830576994 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4002cb10 con 0x7faf480cebb0 2024-04-15T15:19:45.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.005+0000 7faf47400640 1 -- 172.21.15.49:0/830576994 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4002ccb0 con 0x7faf480cebb0 2024-04-15T15:19:45.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.005+0000 7faf47400640 1 -- 172.21.15.49:0/830576994 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4002ce60 con 0x7faf480cebb0 2024-04-15T15:19:45.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.005+0000 7faf4de00640 1 -- 172.21.15.49:0/830576994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 msgr2=0x7faf4806df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.005+0000 7faf4de00640 1 --2- 172.21.15.49:0/830576994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 0x7faf4806df20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faf480d05c0 tx=0x7faf4002bf70 comp rx=0 tx=0).stop 2024-04-15T15:19:45.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.006+0000 7faf4de00640 1 -- 172.21.15.49:0/830576994 shutdown_connections 2024-04-15T15:19:45.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.006+0000 7faf4de00640 1 --2- 172.21.15.49:0/830576994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 0x7faf4806df20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.006+0000 7faf4de00640 1 --2- 172.21.15.49:0/830576994 >> v2:172.21.15.107:3300/0 conn(0x7faf481461c0 0x7faf48071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.006+0000 7faf4de00640 1 -- 172.21.15.49:0/830576994 >> 172.21.15.49:0/830576994 conn(0x7faf4806b930 msgr2=0x7faf4806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:45.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.006+0000 7faf4de00640 1 -- 172.21.15.49:0/830576994 shutdown_connections 2024-04-15T15:19:45.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.006+0000 7faf4de00640 1 -- 172.21.15.49:0/830576994 wait complete. 2024-04-15T15:19:45.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.007+0000 7faf4de00640 1 Processor -- start 2024-04-15T15:19:45.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.007+0000 7faf4de00640 1 -- start start 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.008+0000 7faf4de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf48072420 0x7faf4813a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.008+0000 7faf4de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 0x7faf480b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.008+0000 7faf4de00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faf481461c0 0x7faf480b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.008+0000 7faf4de00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faf480cdb70 con 0x7faf480cebb0 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.008+0000 7faf4de00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf480cd080 con 0x7faf48072420 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.008+0000 7faf4de00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7faf4806e460 con 0x7faf481461c0 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.009+0000 7faf47e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 0x7faf480b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.009+0000 7faf47e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 0x7faf480b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56236/0 (socket says 172.21.15.49:56236) 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.009+0000 7faf47e00640 1 -- 172.21.15.49:0/1343294506 learned_addr learned my addr 172.21.15.49:0/1343294506 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.009+0000 7faf4ca00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf48072420 0x7faf4813a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.009+0000 7faf4d400640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faf481461c0 0x7faf480b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.009+0000 7faf47e00640 1 -- 172.21.15.49:0/1343294506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faf481461c0 msgr2=0x7faf480b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.009+0000 7faf47e00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faf481461c0 0x7faf480b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.009+0000 7faf47e00640 1 -- 172.21.15.49:0/1343294506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf48072420 msgr2=0x7faf4813a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.009+0000 7faf47e00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf48072420 0x7faf4813a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.009+0000 7faf47e00640 1 -- 172.21.15.49:0/1343294506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4813b060 con 0x7faf480cebb0 2024-04-15T15:19:45.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.010+0000 7faf47e00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 0x7faf480b0f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faf3400c400 tx=0x7faf3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.010+0000 7faf44c00640 1 -- 172.21.15.49:0/1343294506 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf340052d0 con 0x7faf480cebb0 2024-04-15T15:19:45.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.010+0000 7faf44c00640 1 -- 172.21.15.49:0/1343294506 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf34005490 con 0x7faf480cebb0 2024-04-15T15:19:45.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.010+0000 7faf44c00640 1 -- 172.21.15.49:0/1343294506 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34007260 con 0x7faf480cebb0 2024-04-15T15:19:45.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.010+0000 7faf4de00640 1 -- 172.21.15.49:0/1343294506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4813b340 con 0x7faf480cebb0 2024-04-15T15:19:45.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.010+0000 7faf4de00640 1 -- 172.21.15.49:0/1343294506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4813b900 con 0x7faf480cebb0 2024-04-15T15:19:45.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.011+0000 7faf4de00640 1 -- 172.21.15.49:0/1343294506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf480755c0 con 0x7faf480cebb0 2024-04-15T15:19:45.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.013+0000 7faf44c00640 1 -- 172.21.15.49:0/1343294506 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf34007400 con 0x7faf480cebb0 2024-04-15T15:19:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.013+0000 7faf44c00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7faf2806c8f0 0x7faf2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.014+0000 7faf44c00640 1 -- 172.21.15.49:0/1343294506 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7faf3408b1c0 con 0x7faf480cebb0 2024-04-15T15:19:45.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.014+0000 7faf4ca00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7faf2806c8f0 0x7faf2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.015+0000 7faf4ca00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7faf2806c8f0 0x7faf2806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faf3c006fd0 tx=0x7faf3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.019+0000 7faf44c00640 1 -- 172.21.15.49:0/1343294506 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf340594d0 con 0x7faf480cebb0 2024-04-15T15:19:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.227+0000 7faf4de00640 1 -- 172.21.15.49:0/1343294506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faf48072200 con 0x7faf480cebb0 2024-04-15T15:19:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.228+0000 7faf44c00640 1 -- 172.21.15.49:0/1343294506 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7faf34058dd0 con 0x7faf480cebb0 2024-04-15T15:19:45.230 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:19:45.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 -- 172.21.15.49:0/1343294506 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7faf2806c8f0 msgr2=0x7faf2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7faf2806c8f0 0x7faf2806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faf3c006fd0 tx=0x7faf3c008040 comp rx=0 tx=0).stop 2024-04-15T15:19:45.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 -- 172.21.15.49:0/1343294506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 msgr2=0x7faf480b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 0x7faf480b0f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faf3400c400 tx=0x7faf3400c8c0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 -- 172.21.15.49:0/1343294506 shutdown_connections 2024-04-15T15:19:45.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7faf2806c8f0 0x7faf2806edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faf481461c0 0x7faf480b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf480cebb0 0x7faf480b0f30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 --2- 172.21.15.49:0/1343294506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf48072420 0x7faf4813a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 -- 172.21.15.49:0/1343294506 >> 172.21.15.49:0/1343294506 conn(0x7faf4806b930 msgr2=0x7faf480b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:45.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 -- 172.21.15.49:0/1343294506 shutdown_connections 2024-04-15T15:19:45.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.231+0000 7faf4de00640 1 -- 172.21.15.49:0/1343294506 wait complete. 2024-04-15T15:19:45.251 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:45.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.470+0000 7f53a2a00640 1 Processor -- start 2024-04-15T15:19:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.470+0000 7f53a2a00640 1 -- start start 2024-04-15T15:19:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.471+0000 7f53a2a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c162020 0x7f539c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.471+0000 7f53a2a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c0d0090 con 0x7f539c0d1c90 2024-04-15T15:19:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.471+0000 7f53a2a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c0cfad0 con 0x7f539c0cf110 2024-04-15T15:19:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.471+0000 7f53a2a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f539c163440 con 0x7f539c162020 2024-04-15T15:19:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.471+0000 7f53a2000640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f539c0d1c90 0x7f539c0d3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47914/0 (socket says 172.21.15.49:47914) 2024-04-15T15:19:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.471+0000 7f53a2000640 1 -- 172.21.15.49:0/1341653386 learned_addr learned my addr 172.21.15.49:0/1341653386 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:45.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.472+0000 7f53a1600640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c162020 0x7f539c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.472+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3261653763 0 0) 0x7f5394001490 con 0x7f539c0d1c90 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.472+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5384003520 con 0x7f539c0d1c90 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.472+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4220460646 0 0) 0x7f5384003520 con 0x7f539c0d1c90 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.472+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c162020 msgr2=0x7f539c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.472+0000 7f539be00640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c162020 0x7f539c162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.472+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 >> v1:172.21.15.107:6789/0 conn(0x7f539c0cf110 legacy=0x7f539c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c0cfdb0 con 0x7f539c0d1c90 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f5394001bc0 con 0x7f539c0d1c90 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 >> v1:172.21.15.49:6789/0 conn(0x7f539c0d1c90 legacy=0x7f539c0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f539be00640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53840039e0 0x7f5384003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f539be00640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c162020 0x7f5384004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f539be00640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5384004d40 0x7f53840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f539c163440 con 0x7f53840039e0 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f538400ba70 con 0x7f5384004d40 2024-04-15T15:19:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f538400bc70 con 0x7f539c162020 2024-04-15T15:19:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f53a1600640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53840039e0 0x7f5384003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f53a1600640 1 -- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c162020 msgr2=0x7f5384004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.473+0000 7f53a0c00640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c162020 0x7f5384004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.474+0000 7f53a2000640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5384004d40 0x7f53840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.474+0000 7f53a1600640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c162020 0x7f5384004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.474+0000 7f53a1600640 1 -- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5384004d40 msgr2=0x7f53840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.474+0000 7f53a1600640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5384004d40 0x7f53840051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.474+0000 7f53a1600640 1 -- 172.21.15.49:0/1341653386 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5384003520 con 0x7f53840039e0 2024-04-15T15:19:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.474+0000 7f53a0c00640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c162020 0x7f5384004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:45.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.475+0000 7f53a1600640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53840039e0 0x7f5384003e00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f538c009a90 tx=0x7f5384005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247951df9ff7b1d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.475+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c039070 con 0x7f53840039e0 2024-04-15T15:19:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.475+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c02c680 con 0x7f53840039e0 2024-04-15T15:19:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.475+0000 7f539be00640 1 -- 172.21.15.49:0/1341653386 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c009690 con 0x7f53840039e0 2024-04-15T15:19:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.476+0000 7f53a2a00640 1 -- 172.21.15.49:0/1341653386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53840039e0 msgr2=0x7f5384003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.476+0000 7f53a2a00640 1 --2- 172.21.15.49:0/1341653386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53840039e0 0x7f5384003e00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f538c009a90 tx=0x7f5384005830 comp rx=0 tx=0).stop 2024-04-15T15:19:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.476+0000 7f53a2000640 1 -- 172.21.15.49:0/1341653386 reap_dead start 2024-04-15T15:19:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.476+0000 7f53a2a00640 1 -- 172.21.15.49:0/1341653386 shutdown_connections 2024-04-15T15:19:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.476+0000 7f53a2a00640 1 -- 172.21.15.49:0/1341653386 >> 172.21.15.49:0/1341653386 conn(0x7f539c06cf00 msgr2=0x7f539c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.476+0000 7f53a2a00640 1 -- 172.21.15.49:0/1341653386 shutdown_connections 2024-04-15T15:19:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.476+0000 7f53a2a00640 1 -- 172.21.15.49:0/1341653386 wait complete. 2024-04-15T15:19:45.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.478+0000 7f53a2a00640 1 Processor -- start 2024-04-15T15:19:45.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.478+0000 7f53a2a00640 1 -- start start 2024-04-15T15:19:45.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.479+0000 7f53a2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53840039e0 0x7f539c0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.479+0000 7f53a2a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f539c0c93a0 0x7f539c0c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.479+0000 7f53a2a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c15e220 0x7f539c15e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.479+0000 7f53a2a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0cf880 con 0x7f539c0c93a0 2024-04-15T15:19:45.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.479+0000 7f53a2a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0b7b30 con 0x7f53840039e0 2024-04-15T15:19:45.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.479+0000 7f53a2a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f539c0d2e70 con 0x7f539c15e220 2024-04-15T15:19:45.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.479+0000 7f53a0c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f539c0c93a0 0x7f539c0c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.479+0000 7f53a0c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f539c0c93a0 0x7f539c0c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56250/0 (socket says 172.21.15.49:56250) 2024-04-15T15:19:45.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.479+0000 7f53a0c00640 1 -- 172.21.15.49:0/3361158502 learned_addr learned my addr 172.21.15.49:0/3361158502 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.480+0000 7f53a1600640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53840039e0 0x7f539c0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.480+0000 7f53a0c00640 1 -- 172.21.15.49:0/3361158502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c15e220 msgr2=0x7f539c15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.480+0000 7f53a2000640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c15e220 0x7f539c15e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.480+0000 7f53a0c00640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c15e220 0x7f539c15e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.480+0000 7f53a0c00640 1 -- 172.21.15.49:0/3361158502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53840039e0 msgr2=0x7f539c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.480+0000 7f53a0c00640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53840039e0 0x7f539c0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.480+0000 7f53a0c00640 1 -- 172.21.15.49:0/3361158502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c0c78c0 con 0x7f539c0c93a0 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.480+0000 7f53a1600640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53840039e0 0x7f539c0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.480+0000 7f53a2000640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c15e220 0x7f539c15e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.480+0000 7f53a0c00640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f539c0c93a0 0x7f539c0c9820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f539000d7a0 tx=0x7f539000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.481+0000 7f5399600640 1 -- 172.21.15.49:0/3361158502 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390005070 con 0x7f539c0c93a0 2024-04-15T15:19:45.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.481+0000 7f5399600640 1 -- 172.21.15.49:0/3361158502 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5390005210 con 0x7f539c0c93a0 2024-04-15T15:19:45.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.481+0000 7f5399600640 1 -- 172.21.15.49:0/3361158502 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390007030 con 0x7f539c0c93a0 2024-04-15T15:19:45.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.481+0000 7f53a2a00640 1 -- 172.21.15.49:0/3361158502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c0c9e60 con 0x7f539c0c93a0 2024-04-15T15:19:45.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.481+0000 7f53a2a00640 1 -- 172.21.15.49:0/3361158502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c0ca420 con 0x7f539c0c93a0 2024-04-15T15:19:45.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.482+0000 7f53a2a00640 1 -- 172.21.15.49:0/3361158502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c15ebf0 con 0x7f539c0c93a0 2024-04-15T15:19:45.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.483+0000 7f5399600640 1 -- 172.21.15.49:0/3361158502 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f539001a030 con 0x7f539c0c93a0 2024-04-15T15:19:45.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.484+0000 7f5399600640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f538006c920 0x7f538006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.485+0000 7f5399600640 1 -- 172.21.15.49:0/3361158502 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f539008b310 con 0x7f539c0c93a0 2024-04-15T15:19:45.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.485+0000 7f53a1600640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f538006c920 0x7f538006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.486+0000 7f53a1600640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f538006c920 0x7f538006ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f538c036000 tx=0x7f538c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.489+0000 7f5399600640 1 -- 172.21.15.49:0/3361158502 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5390021020 con 0x7f539c0c93a0 2024-04-15T15:19:45.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.727+0000 7f53a2a00640 1 -- 172.21.15.49:0/3361158502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f539c0b7650 con 0x7f539c0c93a0 2024-04-15T15:19:45.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.729+0000 7f5399600640 1 -- 172.21.15.49:0/3361158502 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f53900596b0 con 0x7f539c0c93a0 2024-04-15T15:19:45.731 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:45.731 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/203558139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":203558139},{"type":"v1","addr":"172.21.15.107:6833","nonce":203558139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3198352923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3198352923},{"type":"v1","addr":"172.21.15.49:6836","nonce":3198352923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1425337425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1425337425},{"type":"v1","addr":"172.21.15.49:6837","nonce":1425337425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:18:45.325981+0000","modified":"2024-04-15T15:18:48.378286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2374398161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2374398161},{"type":"v1","addr":"172.21.15.107:6835","nonce":2374398161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-15T15:19:45.731 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-04-15T15:19:45.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 -- 172.21.15.49:0/3361158502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f538006c920 msgr2=0x7f538006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f538006c920 0x7f538006ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f538c036000 tx=0x7f538c030000 comp rx=0 tx=0).stop 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 -- 172.21.15.49:0/3361158502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f539c0c93a0 msgr2=0x7f539c0c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f539c0c93a0 0x7f539c0c9820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f539000d7a0 tx=0x7f539000dc60 comp rx=0 tx=0).stop 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 -- 172.21.15.49:0/3361158502 shutdown_connections 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f538006c920 0x7f538006ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f539c15e220 0x7f539c15e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f539c0c93a0 0x7f539c0c9820 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 --2- 172.21.15.49:0/3361158502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53840039e0 0x7f539c0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 -- 172.21.15.49:0/3361158502 >> 172.21.15.49:0/3361158502 conn(0x7f539c06cf00 msgr2=0x7f539c0cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 -- 172.21.15.49:0/3361158502 shutdown_connections 2024-04-15T15:19:45.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.732+0000 7f53a2a00640 1 -- 172.21.15.49:0/3361158502 wait complete. 2024-04-15T15:19:45.751 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:45.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.969+0000 7fa81a800640 1 Processor -- start 2024-04-15T15:19:45.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.969+0000 7fa81a800640 1 -- start start 2024-04-15T15:19:45.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.971+0000 7fa81a800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa8140cee00 0x7fa81406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.971+0000 7fa81a800640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa81415dbb0 con 0x7fa81415d2f0 2024-04-15T15:19:45.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.971+0000 7fa81a800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa81415d8d0 con 0x7fa814071f50 2024-04-15T15:19:45.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.971+0000 7fa81a800640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fa81406f4f0 con 0x7fa8140cee00 2024-04-15T15:19:45.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.971+0000 7fa819400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa8140cee00 0x7fa81406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.971+0000 7fa819400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa8140cee00 0x7fa81406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56262/0 (socket says 172.21.15.49:56262) 2024-04-15T15:19:45.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.971+0000 7fa819400640 1 -- 172.21.15.49:0/1997370894 learned_addr learned my addr 172.21.15.49:0/1997370894 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:45.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.972+0000 7fa819400640 1 -- 172.21.15.49:0/1997370894 >> v1:172.21.15.107:6789/0 conn(0x7fa814071f50 legacy=0x7fa81415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:45.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.972+0000 7fa819400640 1 -- 172.21.15.49:0/1997370894 >> v1:172.21.15.49:6789/0 conn(0x7fa81415d2f0 legacy=0x7fa81415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.972+0000 7fa819400640 1 -- 172.21.15.49:0/1997370894 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81406dde0 con 0x7fa8140cee00 2024-04-15T15:19:45.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.972+0000 7fa813e00640 1 -- 172.21.15.49:0/1997370894 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4095605300 0 0) 0x7fa81415dbb0 con 0x7fa81415d2f0 2024-04-15T15:19:45.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.973+0000 7fa819400640 1 --2- 172.21.15.49:0/1997370894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 0x7fa81406e8c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa808009870 tx=0x7fa80802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d672ea90cb4fbc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.973+0000 7fa813e00640 1 -- 172.21.15.49:0/1997370894 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808034070 con 0x7fa8140cee00 2024-04-15T15:19:45.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.973+0000 7fa813e00640 1 -- 172.21.15.49:0/1997370894 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa808030850 con 0x7fa8140cee00 2024-04-15T15:19:45.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.973+0000 7fa813e00640 1 -- 172.21.15.49:0/1997370894 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808030c20 con 0x7fa8140cee00 2024-04-15T15:19:45.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.974+0000 7fa81a800640 1 -- 172.21.15.49:0/1997370894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 msgr2=0x7fa81406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.974+0000 7fa81a800640 1 --2- 172.21.15.49:0/1997370894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 0x7fa81406e8c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa808009870 tx=0x7fa80802b990 comp rx=0 tx=0).stop 2024-04-15T15:19:45.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.974+0000 7fa81a800640 1 -- 172.21.15.49:0/1997370894 shutdown_connections 2024-04-15T15:19:45.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.974+0000 7fa81a800640 1 --2- 172.21.15.49:0/1997370894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 0x7fa81406e8c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.974+0000 7fa81a800640 1 -- 172.21.15.49:0/1997370894 >> 172.21.15.49:0/1997370894 conn(0x7fa81406b930 msgr2=0x7fa81406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:45.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.974+0000 7fa81a800640 1 -- 172.21.15.49:0/1997370894 shutdown_connections 2024-04-15T15:19:45.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.974+0000 7fa81a800640 1 -- 172.21.15.49:0/1997370894 wait complete. 2024-04-15T15:19:45.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.976+0000 7fa81a800640 1 Processor -- start 2024-04-15T15:19:45.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.976+0000 7fa81a800640 1 -- start start 2024-04-15T15:19:45.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.977+0000 7fa81a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa814071f50 0x7fa814157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.977+0000 7fa81a800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 0x7fa814144a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.977+0000 7fa81a800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa81415d2f0 0x7fa814144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.977+0000 7fa81a800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa8140723b0 con 0x7fa8140cee00 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.977+0000 7fa81a800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa81406e3a0 con 0x7fa814071f50 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.977+0000 7fa81a800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa814072530 con 0x7fa81415d2f0 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.977+0000 7fa818a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 0x7fa814144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.977+0000 7fa818a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 0x7fa814144a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56272/0 (socket says 172.21.15.49:56272) 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.977+0000 7fa818a00640 1 -- 172.21.15.49:0/1451202644 learned_addr learned my addr 172.21.15.49:0/1451202644 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.978+0000 7fa819400640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa814071f50 0x7fa814157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.978+0000 7fa819e00640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa81415d2f0 0x7fa814144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.978+0000 7fa818a00640 1 -- 172.21.15.49:0/1451202644 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa81415d2f0 msgr2=0x7fa814144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.978+0000 7fa818a00640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa81415d2f0 0x7fa814144fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.978+0000 7fa818a00640 1 -- 172.21.15.49:0/1451202644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa814071f50 msgr2=0x7fa814157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.978+0000 7fa818a00640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa814071f50 0x7fa814157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.978+0000 7fa818a00640 1 -- 172.21.15.49:0/1451202644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8140b47d0 con 0x7fa8140cee00 2024-04-15T15:19:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.978+0000 7fa819400640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa814071f50 0x7fa814157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:45.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.978+0000 7fa818a00640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 0x7fa814144a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa80000c460 tx=0x7fa80000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.979+0000 7fa811600640 1 -- 172.21.15.49:0/1451202644 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800016020 con 0x7fa8140cee00 2024-04-15T15:19:45.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.979+0000 7fa811600640 1 -- 172.21.15.49:0/1451202644 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa800012870 con 0x7fa8140cee00 2024-04-15T15:19:45.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.979+0000 7fa811600640 1 -- 172.21.15.49:0/1451202644 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800012c60 con 0x7fa8140cee00 2024-04-15T15:19:45.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.979+0000 7fa81a800640 1 -- 172.21.15.49:0/1451202644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8140b4ab0 con 0x7fa8140cee00 2024-04-15T15:19:45.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.979+0000 7fa81a800640 1 -- 172.21.15.49:0/1451202644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8140b5070 con 0x7fa8140cee00 2024-04-15T15:19:45.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.980+0000 7fa81a800640 1 -- 172.21.15.49:0/1451202644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8140755c0 con 0x7fa8140cee00 2024-04-15T15:19:45.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.981+0000 7fa811600640 1 -- 172.21.15.49:0/1451202644 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa800019020 con 0x7fa8140cee00 2024-04-15T15:19:45.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.982+0000 7fa811600640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa7f406c920 0x7fa7f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:45.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.982+0000 7fa811600640 1 -- 172.21.15.49:0/1451202644 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fa80008bcc0 con 0x7fa8140cee00 2024-04-15T15:19:45.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.982+0000 7fa819400640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa7f406c920 0x7fa7f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:45.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.983+0000 7fa819400640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa7f406c920 0x7fa7f406ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa808036000 tx=0x7fa80803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:45.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:45.987+0000 7fa811600640 1 -- 172.21.15.49:0/1451202644 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa800092080 con 0x7fa8140cee00 2024-04-15T15:19:46.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.190+0000 7fa81a800640 1 -- 172.21.15.49:0/1451202644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8140cdc00 con 0x7fa8140cee00 2024-04-15T15:19:46.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.193+0000 7fa811600640 1 -- 172.21.15.49:0/1451202644 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16854 (secure 0 0 0) 0x7fa800020020 con 0x7fa8140cee00 2024-04-15T15:19:46.195 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:46.196 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:19:32.083681+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:46.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.196+0000 7fa81a800640 1 -- 172.21.15.49:0/1451202644 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa7f406c920 msgr2=0x7fa7f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.196+0000 7fa81a800640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa7f406c920 0x7fa7f406ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa808036000 tx=0x7fa80803b040 comp rx=0 tx=0).stop 2024-04-15T15:19:46.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.196+0000 7fa81a800640 1 -- 172.21.15.49:0/1451202644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 msgr2=0x7fa814144a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.196+0000 7fa81a800640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 0x7fa814144a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa80000c460 tx=0x7fa80000c920 comp rx=0 tx=0).stop 2024-04-15T15:19:46.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.196+0000 7fa81a800640 1 -- 172.21.15.49:0/1451202644 shutdown_connections 2024-04-15T15:19:46.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.196+0000 7fa81a800640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa7f406c920 0x7fa7f406ede0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.196+0000 7fa81a800640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa81415d2f0 0x7fa814144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.196+0000 7fa81a800640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8140cee00 0x7fa814144a60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.196+0000 7fa81a800640 1 --2- 172.21.15.49:0/1451202644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa814071f50 0x7fa814157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.196+0000 7fa81a800640 1 -- 172.21.15.49:0/1451202644 >> 172.21.15.49:0/1451202644 conn(0x7fa81406b930 msgr2=0x7fa8140b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:46.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.197+0000 7fa81a800640 1 -- 172.21.15.49:0/1451202644 shutdown_connections 2024-04-15T15:19:46.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.197+0000 7fa81a800640 1 -- 172.21.15.49:0/1451202644 wait complete. 2024-04-15T15:19:46.218 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-15T15:19:46.219 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:46.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.443+0000 7f23abe00640 1 Processor -- start 2024-04-15T15:19:46.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.444+0000 7f23abe00640 1 -- start start 2024-04-15T15:19:46.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.445+0000 7f23abe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f23ac0738f0 0x7f23ac0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.445+0000 7f23abe00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23ac0b3be0 con 0x7f23ac0b37d0 2024-04-15T15:19:46.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.445+0000 7f23abe00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23ac159300 con 0x7f23ac158f20 2024-04-15T15:19:46.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.445+0000 7f23abe00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f23ac0cf600 con 0x7f23ac0738f0 2024-04-15T15:19:46.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.446+0000 7f23aa000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f23ac0738f0 0x7f23ac0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.446+0000 7f23ab400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f23ac0b37d0 0x7f23ac0d06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47938/0 (socket says 172.21.15.49:47938) 2024-04-15T15:19:46.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.446+0000 7f23ab400640 1 -- 172.21.15.49:0/1046802342 learned_addr learned my addr 172.21.15.49:0/1046802342 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:46.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.447+0000 7f23aa000640 1 -- 172.21.15.49:0/1046802342 >> v1:172.21.15.107:6789/0 conn(0x7f23ac158f20 legacy=0x7f23ac160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.447+0000 7f23aa000640 1 -- 172.21.15.49:0/1046802342 >> v1:172.21.15.49:6789/0 conn(0x7f23ac0b37d0 legacy=0x7f23ac0d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.447+0000 7f23aa000640 1 -- 172.21.15.49:0/1046802342 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac0cf8e0 con 0x7f23ac0738f0 2024-04-15T15:19:46.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.448+0000 7f23aa000640 1 --2- 172.21.15.49:0/1046802342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 0x7f23ac0b3290 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f23940097a0 tx=0x7f239402b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86abc46c532c60e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:46.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.448+0000 7f23a9600640 1 -- 172.21.15.49:0/1046802342 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394037070 con 0x7f23ac0738f0 2024-04-15T15:19:46.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.448+0000 7f23a9600640 1 -- 172.21.15.49:0/1046802342 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2394030710 con 0x7f23ac0738f0 2024-04-15T15:19:46.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.448+0000 7f23a9600640 1 -- 172.21.15.49:0/1046802342 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394030ae0 con 0x7f23ac0738f0 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.449+0000 7f23abe00640 1 -- 172.21.15.49:0/1046802342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 msgr2=0x7f23ac0b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.449+0000 7f23abe00640 1 --2- 172.21.15.49:0/1046802342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 0x7f23ac0b3290 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f23940097a0 tx=0x7f239402b710 comp rx=0 tx=0).stop 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.449+0000 7f23abe00640 1 -- 172.21.15.49:0/1046802342 shutdown_connections 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.449+0000 7f23abe00640 1 --2- 172.21.15.49:0/1046802342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 0x7f23ac0b3290 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.449+0000 7f23abe00640 1 -- 172.21.15.49:0/1046802342 >> 172.21.15.49:0/1046802342 conn(0x7f23ac067720 msgr2=0x7f23ac067b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.449+0000 7f23abe00640 1 -- 172.21.15.49:0/1046802342 shutdown_connections 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.449+0000 7f23abe00640 1 -- 172.21.15.49:0/1046802342 wait complete. 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.451+0000 7f23abe00640 1 Processor -- start 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.451+0000 7f23abe00640 1 -- start start 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.452+0000 7f23abe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 0x7f23ac1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.452+0000 7f23abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23ac0b37d0 0x7f23ac160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.452+0000 7f23abe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23ac158f20 0x7f23ac06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.452+0000 7f23abe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac0d0180 con 0x7f23ac0738f0 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.452+0000 7f23abe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac0cfea0 con 0x7f23ac0b37d0 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.452+0000 7f23abe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f23ac0cf2b0 con 0x7f23ac158f20 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.452+0000 7f23aaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 0x7f23ac1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.452+0000 7f23aaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 0x7f23ac1539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56288/0 (socket says 172.21.15.49:56288) 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.452+0000 7f23aaa00640 1 -- 172.21.15.49:0/2460629806 learned_addr learned my addr 172.21.15.49:0/2460629806 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.453+0000 7f23aa000640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23ac0b37d0 0x7f23ac160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.453+0000 7f23aaa00640 1 -- 172.21.15.49:0/2460629806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23ac158f20 msgr2=0x7f23ac06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.453+0000 7f23ab400640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23ac158f20 0x7f23ac06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.453+0000 7f23aaa00640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23ac158f20 0x7f23ac06ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.453+0000 7f23aaa00640 1 -- 172.21.15.49:0/2460629806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23ac0b37d0 msgr2=0x7f23ac160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.453+0000 7f23aaa00640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23ac0b37d0 0x7f23ac160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.453+0000 7f23aaa00640 1 -- 172.21.15.49:0/2460629806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac1f8390 con 0x7f23ac0738f0 2024-04-15T15:19:46.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.453+0000 7f23aa000640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23ac0b37d0 0x7f23ac160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:46.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.453+0000 7f23aaa00640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 0x7f23ac1539b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f239c00c3c0 tx=0x7f239c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:46.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.453+0000 7f23a3400640 1 -- 172.21.15.49:0/2460629806 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c0052d0 con 0x7f23ac0738f0 2024-04-15T15:19:46.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.454+0000 7f23a3400640 1 -- 172.21.15.49:0/2460629806 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239c005490 con 0x7f23ac0738f0 2024-04-15T15:19:46.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.454+0000 7f23a3400640 1 -- 172.21.15.49:0/2460629806 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c007260 con 0x7f23ac0738f0 2024-04-15T15:19:46.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.454+0000 7f23abe00640 1 -- 172.21.15.49:0/2460629806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac1f8670 con 0x7f23ac0738f0 2024-04-15T15:19:46.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.454+0000 7f23abe00640 1 -- 172.21.15.49:0/2460629806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac1f8c30 con 0x7f23ac0738f0 2024-04-15T15:19:46.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.455+0000 7f23abe00640 1 -- 172.21.15.49:0/2460629806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ac1f8e70 con 0x7f23ac0738f0 2024-04-15T15:19:46.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.456+0000 7f23a3400640 1 -- 172.21.15.49:0/2460629806 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f239c007400 con 0x7f23ac0738f0 2024-04-15T15:19:46.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.457+0000 7f23a3400640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f238806c8f0 0x7f238806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.457+0000 7f23a3400640 1 -- 172.21.15.49:0/2460629806 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f239c08b0a0 con 0x7f23ac0738f0 2024-04-15T15:19:46.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.458+0000 7f23aa000640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f238806c8f0 0x7f238806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.458+0000 7f23aa000640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f238806c8f0 0x7f238806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2394034000 tx=0x7f2394008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:46.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.462+0000 7f23a3400640 1 -- 172.21.15.49:0/2460629806 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f239c059440 con 0x7f23ac0738f0 2024-04-15T15:19:46.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.705+0000 7f23abe00640 1 -- 172.21.15.49:0/2460629806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23ac1569d0 con 0x7f23ac0738f0 2024-04-15T15:19:46.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.707+0000 7f23a3400640 1 -- 172.21.15.49:0/2460629806 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f239c058d40 con 0x7f23ac0738f0 2024-04-15T15:19:46.709 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:46.709 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/203558139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":203558139},{"type":"v1","addr":"172.21.15.107:6833","nonce":203558139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3198352923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3198352923},{"type":"v1","addr":"172.21.15.49:6836","nonce":3198352923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1425337425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1425337425},{"type":"v1","addr":"172.21.15.49:6837","nonce":1425337425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:18:45.325981+0000","modified":"2024-04-15T15:18:48.378286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2374398161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2374398161},{"type":"v1","addr":"172.21.15.107:6835","nonce":2374398161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-15T15:19:46.709 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.709+0000 7f23abe00640 1 -- 172.21.15.49:0/2460629806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f238806c8f0 msgr2=0x7f238806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f238806c8f0 0x7f238806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2394034000 tx=0x7f2394008030 comp rx=0 tx=0).stop 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 -- 172.21.15.49:0/2460629806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 msgr2=0x7f23ac1539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 0x7f23ac1539b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f239c00c3c0 tx=0x7f239c00c880 comp rx=0 tx=0).stop 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 -- 172.21.15.49:0/2460629806 shutdown_connections 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f238806c8f0 0x7f238806edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23ac158f20 0x7f23ac06ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23ac0b37d0 0x7f23ac160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 --2- 172.21.15.49:0/2460629806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23ac0738f0 0x7f23ac1539b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 -- 172.21.15.49:0/2460629806 >> 172.21.15.49:0/2460629806 conn(0x7f23ac067720 msgr2=0x7f23ac1557b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 -- 172.21.15.49:0/2460629806 shutdown_connections 2024-04-15T15:19:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.710+0000 7f23abe00640 1 -- 172.21.15.49:0/2460629806 wait complete. 2024-04-15T15:19:46.733 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:19:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.957+0000 7f268ae00640 1 Processor -- start 2024-04-15T15:19:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.958+0000 7f268ae00640 1 -- start start 2024-04-15T15:19:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.959+0000 7f268ae00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2684162020 0x7f2684162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.959+0000 7f268ae00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26840d0090 con 0x7f26840cf110 2024-04-15T15:19:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.959+0000 7f268ae00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26840cfad0 con 0x7f26840d1c90 2024-04-15T15:19:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.959+0000 7f268ae00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f2684163440 con 0x7f2684162020 2024-04-15T15:19:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.960+0000 7f2689000640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f26840cf110 0x7f26840d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47946/0 (socket says 172.21.15.49:47946) 2024-04-15T15:19:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.960+0000 7f2689000640 1 -- 172.21.15.49:0/242305934 learned_addr learned my addr 172.21.15.49:0/242305934 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:46.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.960+0000 7f2689a00640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2684162020 0x7f2684162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.960+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1379725845 0 0) 0x7f2674001490 con 0x7f26840cf110 2024-04-15T15:19:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.960+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f266c003520 con 0x7f26840cf110 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407463289 0 0) 0x7f26840cfad0 con 0x7f26840d1c90 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2674001490 con 0x7f26840d1c90 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2753460252 0 0) 0x7f266c003520 con 0x7f26840cf110 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2684162020 msgr2=0x7f2684162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2684162020 0x7f2684162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 >> v1:172.21.15.107:6789/0 conn(0x7f26840d1c90 legacy=0x7f26840d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26840cfdb0 con 0x7f26840cf110 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f2674001b40 con 0x7f26840cf110 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 >> v1:172.21.15.49:6789/0 conn(0x7f26840cf110 legacy=0x7f26840d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2684162020 0x7f266c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f266c004120 0x7f266c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f266c004d40 0x7f266c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2684163440 con 0x7f266c004d40 2024-04-15T15:19:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f266c00bb00 con 0x7f266c004120 2024-04-15T15:19:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f266c00bd00 con 0x7f2684162020 2024-04-15T15:19:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.961+0000 7f268a400640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f266c004d40 0x7f266c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.962+0000 7f2689000640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f266c004120 0x7f266c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.962+0000 7f2689a00640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2684162020 0x7f266c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.962+0000 7f268a400640 1 -- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2684162020 msgr2=0x7f266c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.962+0000 7f268a400640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2684162020 0x7f266c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.962+0000 7f268a400640 1 -- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f266c004120 msgr2=0x7f266c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.962+0000 7f268a400640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f266c004120 0x7f266c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.962+0000 7f268a400640 1 -- 172.21.15.49:0/242305934 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c003520 con 0x7f266c004d40 2024-04-15T15:19:46.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.963+0000 7f268a400640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f266c004d40 0x7f266c0051c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f267c0018d0 tx=0x7f266c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fbf84ea2e95a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.963+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c037070 con 0x7f266c004d40 2024-04-15T15:19:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.963+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f267c0052d0 con 0x7f266c004d40 2024-04-15T15:19:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.963+0000 7f2683e00640 1 -- 172.21.15.49:0/242305934 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c0014f0 con 0x7f266c004d40 2024-04-15T15:19:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.964+0000 7f268ae00640 1 -- 172.21.15.49:0/242305934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f266c004d40 msgr2=0x7f266c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.964+0000 7f268ae00640 1 --2- 172.21.15.49:0/242305934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f266c004d40 0x7f266c0051c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f267c0018d0 tx=0x7f266c005830 comp rx=0 tx=0).stop 2024-04-15T15:19:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.964+0000 7f268a400640 1 -- 172.21.15.49:0/242305934 reap_dead start 2024-04-15T15:19:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.964+0000 7f268ae00640 1 -- 172.21.15.49:0/242305934 shutdown_connections 2024-04-15T15:19:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.964+0000 7f268ae00640 1 -- 172.21.15.49:0/242305934 >> 172.21.15.49:0/242305934 conn(0x7f268406cf00 msgr2=0x7f268406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.964+0000 7f268ae00640 1 -- 172.21.15.49:0/242305934 shutdown_connections 2024-04-15T15:19:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.964+0000 7f268ae00640 1 -- 172.21.15.49:0/242305934 wait complete. 2024-04-15T15:19:46.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.966+0000 7f268ae00640 1 Processor -- start 2024-04-15T15:19:46.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.966+0000 7f268ae00640 1 -- start start 2024-04-15T15:19:46.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.967+0000 7f268ae00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26840ca850 0x7f268415f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.967+0000 7f268ae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f268415fe40 0x7f2684147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.967+0000 7f268ae00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26841598d0 0x7f2684148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.967+0000 7f268ae00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f26840cf880 con 0x7f26840ca850 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.967+0000 7f268ae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f26840b7b30 con 0x7f268415fe40 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.967+0000 7f268ae00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f26840d2e70 con 0x7f26841598d0 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.967+0000 7f2689a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26840ca850 0x7f268415f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.967+0000 7f2689a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26840ca850 0x7f268415f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56300/0 (socket says 172.21.15.49:56300) 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.967+0000 7f2689a00640 1 -- 172.21.15.49:0/474459 learned_addr learned my addr 172.21.15.49:0/474459 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.968+0000 7f2689000640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f268415fe40 0x7f2684147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.968+0000 7f2689a00640 1 -- 172.21.15.49:0/474459 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26841598d0 msgr2=0x7f2684148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.968+0000 7f2689a00640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26841598d0 0x7f2684148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.968+0000 7f2689a00640 1 -- 172.21.15.49:0/474459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f268415fe40 msgr2=0x7f2684147ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.968+0000 7f2689a00640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f268415fe40 0x7f2684147ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.968+0000 7f2689a00640 1 -- 172.21.15.49:0/474459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2684146740 con 0x7f26840ca850 2024-04-15T15:19:46.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.968+0000 7f2689000640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f268415fe40 0x7f2684147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:46.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.968+0000 7f2689a00640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26840ca850 0x7f268415f900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f26780115b0 tx=0x7f2678011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:46.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.968+0000 7f2681600640 1 -- 172.21.15.49:0/474459 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678015870 con 0x7f26840ca850 2024-04-15T15:19:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.969+0000 7f2681600640 1 -- 172.21.15.49:0/474459 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2678015a10 con 0x7f26840ca850 2024-04-15T15:19:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.969+0000 7f2681600640 1 -- 172.21.15.49:0/474459 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678015dc0 con 0x7f26840ca850 2024-04-15T15:19:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.969+0000 7f268ae00640 1 -- 172.21.15.49:0/474459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2684146a20 con 0x7f26840ca850 2024-04-15T15:19:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.969+0000 7f268ae00640 1 -- 172.21.15.49:0/474459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2684146fe0 con 0x7f26840ca850 2024-04-15T15:19:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.970+0000 7f268ae00640 1 -- 172.21.15.49:0/474459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26840755c0 con 0x7f26840ca850 2024-04-15T15:19:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.971+0000 7f2681600640 1 -- 172.21.15.49:0/474459 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f267800f480 con 0x7f26840ca850 2024-04-15T15:19:46.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.972+0000 7f2681600640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f266806c8f0 0x7f266806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:46.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.972+0000 7f2681600640 1 -- 172.21.15.49:0/474459 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f267808c610 con 0x7f26840ca850 2024-04-15T15:19:46.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.972+0000 7f2689000640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f266806c8f0 0x7f266806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:46.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.973+0000 7f2689000640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f266806c8f0 0x7f266806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2674002230 tx=0x7f2674002120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:46.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:46.976+0000 7f2681600640 1 -- 172.21.15.49:0/474459 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2678019030 con 0x7f26840ca850 2024-04-15T15:19:47.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.221+0000 7f268ae00640 1 -- 172.21.15.49:0/474459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26840b7650 con 0x7f26840ca850 2024-04-15T15:19:47.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.222+0000 7f2681600640 1 -- 172.21.15.49:0/474459 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f267805a890 con 0x7f26840ca850 2024-04-15T15:19:47.225 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:47.225 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/203558139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":203558139},{"type":"v1","addr":"172.21.15.107:6833","nonce":203558139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3198352923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3198352923},{"type":"v1","addr":"172.21.15.49:6836","nonce":3198352923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1425337425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1425337425},{"type":"v1","addr":"172.21.15.49:6837","nonce":1425337425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:18:45.325981+0000","modified":"2024-04-15T15:18:48.378286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2374398161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2374398161},{"type":"v1","addr":"172.21.15.107:6835","nonce":2374398161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-15T15:19:47.226 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-04-15T15:19:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.225+0000 7f268ae00640 1 -- 172.21.15.49:0/474459 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f266806c8f0 msgr2=0x7f266806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.225+0000 7f268ae00640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f266806c8f0 0x7f266806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2674002230 tx=0x7f2674002120 comp rx=0 tx=0).stop 2024-04-15T15:19:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.225+0000 7f268ae00640 1 -- 172.21.15.49:0/474459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26840ca850 msgr2=0x7f268415f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.225+0000 7f268ae00640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26840ca850 0x7f268415f900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f26780115b0 tx=0x7f2678011a70 comp rx=0 tx=0).stop 2024-04-15T15:19:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.225+0000 7f268ae00640 1 -- 172.21.15.49:0/474459 shutdown_connections 2024-04-15T15:19:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.225+0000 7f268ae00640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f266806c8f0 0x7f266806edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.225+0000 7f268ae00640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26841598d0 0x7f2684148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.225+0000 7f268ae00640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f268415fe40 0x7f2684147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.225+0000 7f268ae00640 1 --2- 172.21.15.49:0/474459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26840ca850 0x7f268415f900 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.225+0000 7f268ae00640 1 -- 172.21.15.49:0/474459 >> 172.21.15.49:0/474459 conn(0x7f268406cf00 msgr2=0x7f26840cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:47.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.226+0000 7f268ae00640 1 -- 172.21.15.49:0/474459 shutdown_connections 2024-04-15T15:19:47.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.226+0000 7f268ae00640 1 -- 172.21.15.49:0/474459 wait complete. 2024-04-15T15:19:47.249 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:47.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.486+0000 7fdea7e00640 1 Processor -- start 2024-04-15T15:19:47.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.486+0000 7fdea7e00640 1 -- start start 2024-04-15T15:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.487+0000 7fdea7e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdea8162020 0x7fdea8162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.487+0000 7fdea7e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdea80cf110 0x7fdea80d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.487+0000 7fdea7e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80d1ac0 0x7fdea80b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.487+0000 7fdea7e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fdea80d1f50 con 0x7fdea80cf110 2024-04-15T15:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.488+0000 7fdea7e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fdea8163440 con 0x7fdea8162020 2024-04-15T15:19:47.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.488+0000 7fdea7e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdea80b7b30 con 0x7fdea80d1ac0 2024-04-15T15:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.488+0000 7fdea6000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdea80cf110 0x7fdea80d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.488+0000 7fdea6000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdea80cf110 0x7fdea80d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56310/0 (socket says 172.21.15.49:56310) 2024-04-15T15:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.488+0000 7fdea6000640 1 -- 172.21.15.49:0/2425597352 learned_addr learned my addr 172.21.15.49:0/2425597352 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.488+0000 7fdea6a00640 1 --2- 172.21.15.49:0/2425597352 >> v2:172.21.15.107:3300/0 conn(0x7fdea8162020 0x7fdea8162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.488+0000 7fdea7400640 1 --2- 172.21.15.49:0/2425597352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80d1ac0 0x7fdea80b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.488+0000 7fdea6000640 1 -- 172.21.15.49:0/2425597352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80d1ac0 msgr2=0x7fdea80b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.488+0000 7fdea6000640 1 --2- 172.21.15.49:0/2425597352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80d1ac0 0x7fdea80b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.489+0000 7fdea6000640 1 -- 172.21.15.49:0/2425597352 >> v2:172.21.15.107:3300/0 conn(0x7fdea8162020 msgr2=0x7fdea8162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.489+0000 7fdea6000640 1 --2- 172.21.15.49:0/2425597352 >> v2:172.21.15.107:3300/0 conn(0x7fdea8162020 0x7fdea8162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.489+0000 7fdea6000640 1 -- 172.21.15.49:0/2425597352 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea80d0090 con 0x7fdea80cf110 2024-04-15T15:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.490+0000 7fdea6000640 1 --2- 172.21.15.49:0/2425597352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea80cf110 0x7fdea80d1580 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdea808cd90 tx=0x7fde9002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587b13380e6a978b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.490+0000 7fdea5600640 1 -- 172.21.15.49:0/2425597352 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90035020 con 0x7fdea80cf110 2024-04-15T15:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.490+0000 7fdea5600640 1 -- 172.21.15.49:0/2425597352 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde90031870 con 0x7fdea80cf110 2024-04-15T15:19:47.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.490+0000 7fdea5600640 1 -- 172.21.15.49:0/2425597352 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90031c40 con 0x7fdea80cf110 2024-04-15T15:19:47.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.491+0000 7fdea7e00640 1 -- 172.21.15.49:0/2425597352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea80cf110 msgr2=0x7fdea80d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.491+0000 7fdea7e00640 1 --2- 172.21.15.49:0/2425597352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea80cf110 0x7fdea80d1580 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdea808cd90 tx=0x7fde9002b910 comp rx=0 tx=0).stop 2024-04-15T15:19:47.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.491+0000 7fdea7e00640 1 -- 172.21.15.49:0/2425597352 shutdown_connections 2024-04-15T15:19:47.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.491+0000 7fdea7e00640 1 --2- 172.21.15.49:0/2425597352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80d1ac0 0x7fdea80b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.491+0000 7fdea7e00640 1 --2- 172.21.15.49:0/2425597352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea80cf110 0x7fdea80d1580 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.491+0000 7fdea7e00640 1 --2- 172.21.15.49:0/2425597352 >> v2:172.21.15.107:3300/0 conn(0x7fdea8162020 0x7fdea8162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.491+0000 7fdea7e00640 1 -- 172.21.15.49:0/2425597352 >> 172.21.15.49:0/2425597352 conn(0x7fdea806cf00 msgr2=0x7fdea806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:47.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.491+0000 7fdea7e00640 1 -- 172.21.15.49:0/2425597352 shutdown_connections 2024-04-15T15:19:47.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.491+0000 7fdea7e00640 1 -- 172.21.15.49:0/2425597352 wait complete. 2024-04-15T15:19:47.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.493+0000 7fdea7e00640 1 Processor -- start 2024-04-15T15:19:47.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.493+0000 7fdea7e00640 1 -- start start 2024-04-15T15:19:47.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.494+0000 7fdea7e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80cf110 0x7fdea8155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.494+0000 7fdea7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea80d1ac0 0x7fdea8155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.494+0000 7fdea7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea8162020 0x7fdea80c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.494+0000 7fdea7e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdea80cfdb0 con 0x7fdea8162020 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.494+0000 7fdea7e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdea80cf880 con 0x7fdea80d1ac0 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.494+0000 7fdea7e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdea80d2b90 con 0x7fdea80cf110 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.494+0000 7fdea7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea8162020 0x7fdea80c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea8162020 0x7fdea80c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56316/0 (socket says 172.21.15.49:56316) 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea7400640 1 -- 172.21.15.49:0/728640332 learned_addr learned my addr 172.21.15.49:0/728640332 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea6a00640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80cf110 0x7fdea8155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea6000640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea80d1ac0 0x7fdea8155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea7400640 1 -- 172.21.15.49:0/728640332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80cf110 msgr2=0x7fdea8155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea7400640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80cf110 0x7fdea8155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea7400640 1 -- 172.21.15.49:0/728640332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea80d1ac0 msgr2=0x7fdea8155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea7400640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea80d1ac0 0x7fdea8155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea7400640 1 -- 172.21.15.49:0/728640332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea8156090 con 0x7fdea8162020 2024-04-15T15:19:47.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea6a00640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80cf110 0x7fdea8155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:47.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.495+0000 7fdea7400640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea8162020 0x7fdea80c93a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdea0007e60 tx=0x7fdea000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.496+0000 7fde9f400640 1 -- 172.21.15.49:0/728640332 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0014620 con 0x7fdea8162020 2024-04-15T15:19:47.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.496+0000 7fde9f400640 1 -- 172.21.15.49:0/728640332 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea0019070 con 0x7fdea8162020 2024-04-15T15:19:47.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.496+0000 7fde9f400640 1 -- 172.21.15.49:0/728640332 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0014a20 con 0x7fdea8162020 2024-04-15T15:19:47.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.496+0000 7fdea7e00640 1 -- 172.21.15.49:0/728640332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea8156370 con 0x7fdea8162020 2024-04-15T15:19:47.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.496+0000 7fdea7e00640 1 -- 172.21.15.49:0/728640332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea8155d90 con 0x7fdea8162020 2024-04-15T15:19:47.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.497+0000 7fdea7e00640 1 -- 172.21.15.49:0/728640332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea8156890 con 0x7fdea8162020 2024-04-15T15:19:47.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.498+0000 7fde9f400640 1 -- 172.21.15.49:0/728640332 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdea0014bc0 con 0x7fdea8162020 2024-04-15T15:19:47.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.499+0000 7fde9f400640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fde8406c8f0 0x7fde8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.499+0000 7fde9f400640 1 -- 172.21.15.49:0/728640332 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fdea008c4e0 con 0x7fdea8162020 2024-04-15T15:19:47.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.499+0000 7fdea6a00640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fde8406c8f0 0x7fde8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.500+0000 7fdea6a00640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fde8406c8f0 0x7fde8406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fde98006fd0 tx=0x7fde98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.504+0000 7fde9f400640 1 -- 172.21.15.49:0/728640332 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdea005a890 con 0x7fdea8162020 2024-04-15T15:19:47.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.711+0000 7fdea7e00640 1 -- 172.21.15.49:0/728640332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdea80755c0 con 0x7fdea8162020 2024-04-15T15:19:47.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.713+0000 7fde9f400640 1 -- 172.21.15.49:0/728640332 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16854 (secure 0 0 0) 0x7fdea005a190 con 0x7fdea8162020 2024-04-15T15:19:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:47.717 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:19:32.083681+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:18:43.124453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:18:44.307843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:47.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 -- 172.21.15.49:0/728640332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fde8406c8f0 msgr2=0x7fde8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fde8406c8f0 0x7fde8406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fde98006fd0 tx=0x7fde98008040 comp rx=0 tx=0).stop 2024-04-15T15:19:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 -- 172.21.15.49:0/728640332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea8162020 msgr2=0x7fdea80c93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea8162020 0x7fdea80c93a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdea0007e60 tx=0x7fdea000e4b0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 -- 172.21.15.49:0/728640332 shutdown_connections 2024-04-15T15:19:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fde8406c8f0 0x7fde8406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdea8162020 0x7fdea80c93a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea80d1ac0 0x7fdea8155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 --2- 172.21.15.49:0/728640332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdea80cf110 0x7fdea8155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 -- 172.21.15.49:0/728640332 >> 172.21.15.49:0/728640332 conn(0x7fdea806cf00 msgr2=0x7fdea80cbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:47.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 -- 172.21.15.49:0/728640332 shutdown_connections 2024-04-15T15:19:47.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.716+0000 7fdea7e00640 1 -- 172.21.15.49:0/728640332 wait complete. 2024-04-15T15:19:47.741 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-15T15:19:47.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.970+0000 7f7e9b200640 1 Processor -- start 2024-04-15T15:19:47.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.970+0000 7f7e9b200640 1 -- start start 2024-04-15T15:19:47.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.971+0000 7f7e9b200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e9c15b410 0x7f7e9c15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.971+0000 7f7e9b200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7e9c0cf530 0x7f7e9c0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.971+0000 7f7e9b200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e9c15cae0 con 0x7f7e9c15f110 2024-04-15T15:19:47.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.971+0000 7f7e9b200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7e9c0cd360 con 0x7f7e9c0cf530 2024-04-15T15:19:47.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.971+0000 7f7e9b200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9c0cd640 con 0x7f7e9c15b410 2024-04-15T15:19:47.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.972+0000 7f7e99400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7e9c0cf530 0x7f7e9c0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.972+0000 7f7e99400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7e9c0cf530 0x7f7e9c0cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56328/0 (socket says 172.21.15.49:56328) 2024-04-15T15:19:47.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.972+0000 7f7e99400640 1 -- 172.21.15.49:0/292277331 learned_addr learned my addr 172.21.15.49:0/292277331 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:47.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.972+0000 7f7e99e00640 1 --2- 172.21.15.49:0/292277331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e9c15b410 0x7f7e9c15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.972+0000 7f7e99400640 1 -- 172.21.15.49:0/292277331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e9c15b410 msgr2=0x7f7e9c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:47.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.972+0000 7f7e99400640 1 --2- 172.21.15.49:0/292277331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e9c15b410 0x7f7e9c15c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.972+0000 7f7e99400640 1 -- 172.21.15.49:0/292277331 >> v1:172.21.15.49:6789/0 conn(0x7f7e9c15f110 legacy=0x7f7e9c15f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.972+0000 7f7e98a00640 1 -- 172.21.15.49:0/292277331 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 561345894 0 0) 0x7f7e94001490 con 0x7f7e9c15f110 2024-04-15T15:19:47.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.972+0000 7f7e99400640 1 -- 172.21.15.49:0/292277331 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c0cd920 con 0x7f7e9c0cf530 2024-04-15T15:19:47.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.973+0000 7f7e99400640 1 --2- 172.21.15.49:0/292277331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c0cf530 0x7f7e9c0cf9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400bb10 tx=0x7f7e8402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d7db14518137d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.974+0000 7f7e98a00640 1 -- 172.21.15.49:0/292277331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8403b070 con 0x7f7e9c0cf530 2024-04-15T15:19:47.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.974+0000 7f7e98a00640 1 -- 172.21.15.49:0/292277331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e84036440 con 0x7f7e9c0cf530 2024-04-15T15:19:47.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.974+0000 7f7e98a00640 1 -- 172.21.15.49:0/292277331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84036810 con 0x7f7e9c0cf530 2024-04-15T15:19:47.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.974+0000 7f7e9b200640 1 -- 172.21.15.49:0/292277331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c0cf530 msgr2=0x7f7e9c0cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.974+0000 7f7e9b200640 1 --2- 172.21.15.49:0/292277331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c0cf530 0x7f7e9c0cf9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400bb10 tx=0x7f7e8402e0c0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.975+0000 7f7e9b200640 1 -- 172.21.15.49:0/292277331 shutdown_connections 2024-04-15T15:19:47.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.975+0000 7f7e9b200640 1 --2- 172.21.15.49:0/292277331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c0cf530 0x7f7e9c0cf9c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.975+0000 7f7e9b200640 1 --2- 172.21.15.49:0/292277331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e9c15b410 0x7f7e9c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.975+0000 7f7e9b200640 1 -- 172.21.15.49:0/292277331 >> 172.21.15.49:0/292277331 conn(0x7f7e9c0b23a0 msgr2=0x7f7e9c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:47.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.975+0000 7f7e9b200640 1 -- 172.21.15.49:0/292277331 shutdown_connections 2024-04-15T15:19:47.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.975+0000 7f7e9b200640 1 -- 172.21.15.49:0/292277331 wait complete. 2024-04-15T15:19:47.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.977+0000 7f7e9b200640 1 Processor -- start 2024-04-15T15:19:47.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.977+0000 7f7e9b200640 1 -- start start 2024-04-15T15:19:47.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e9b200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e9c0cf530 0x7f7e9c141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e9b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c15b410 0x7f7e9c1f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e9b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e9c15f110 0x7f7e9c15dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e9b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c0cff00 con 0x7f7e9c15b410 2024-04-15T15:19:47.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e9b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c0cdee0 con 0x7f7e9c15f110 2024-04-15T15:19:47.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e9b200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9c0cd030 con 0x7f7e9c0cf530 2024-04-15T15:19:47.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e99400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c15b410 0x7f7e9c1f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e99400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c15b410 0x7f7e9c1f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56344/0 (socket says 172.21.15.49:56344) 2024-04-15T15:19:47.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e99400640 1 -- 172.21.15.49:0/2753256290 learned_addr learned my addr 172.21.15.49:0/2753256290 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:47.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e99400640 1 -- 172.21.15.49:0/2753256290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e9c0cf530 msgr2=0x7f7e9c141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e99400640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e9c0cf530 0x7f7e9c141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e99400640 1 -- 172.21.15.49:0/2753256290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e9c15f110 msgr2=0x7f7e9c15dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:47.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e99400640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e9c15f110 0x7f7e9c15dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:47.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.978+0000 7f7e99400640 1 -- 172.21.15.49:0/2753256290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c15ac10 con 0x7f7e9c15b410 2024-04-15T15:19:47.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.979+0000 7f7e99400640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c15b410 0x7f7e9c1f4390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7e8402e100 tx=0x7f7e840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.979+0000 7f7e92000640 1 -- 172.21.15.49:0/2753256290 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84045020 con 0x7f7e9c15b410 2024-04-15T15:19:47.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.979+0000 7f7e92000640 1 -- 172.21.15.49:0/2753256290 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e840346e0 con 0x7f7e9c15b410 2024-04-15T15:19:47.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.979+0000 7f7e9b200640 1 -- 172.21.15.49:0/2753256290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c0b3530 con 0x7f7e9c15b410 2024-04-15T15:19:47.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.979+0000 7f7e92000640 1 -- 172.21.15.49:0/2753256290 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8403b040 con 0x7f7e9c15b410 2024-04-15T15:19:47.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.979+0000 7f7e9b200640 1 -- 172.21.15.49:0/2753256290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c1567b0 con 0x7f7e9c15b410 2024-04-15T15:19:47.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.980+0000 7f7e9b200640 1 -- 172.21.15.49:0/2753256290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9c0755c0 con 0x7f7e9c15b410 2024-04-15T15:19:47.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.981+0000 7f7e92000640 1 -- 172.21.15.49:0/2753256290 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e84048020 con 0x7f7e9c15b410 2024-04-15T15:19:47.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.982+0000 7f7e92000640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7e7806c8f0 0x7f7e7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:47.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.983+0000 7f7e92000640 1 -- 172.21.15.49:0/2753256290 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f7e840b0690 con 0x7f7e9c15b410 2024-04-15T15:19:47.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.983+0000 7f7e99e00640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7e7806c8f0 0x7f7e7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:47.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.983+0000 7f7e99e00640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7e7806c8f0 0x7f7e7806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0098f0 tx=0x7f7e8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:47.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:47.987+0000 7f7e92000640 1 -- 172.21.15.49:0/2753256290 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e8404f030 con 0x7f7e9c15b410 2024-04-15T15:19:48.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.233+0000 7f7e9b200640 1 -- 172.21.15.49:0/2753256290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7e9c15bad0 con 0x7f7e9c15b410 2024-04-15T15:19:48.236 INFO:tasks.ceph.mon.a.smithi049.stderr:2024-04-15T15:19:48.234+0000 7fbf18400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.238+0000 7f7e92000640 1 -- 172.21.15.49:0/2753256290 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f7e8407ea30 con 0x7f7e9c15b410 2024-04-15T15:19:48.241 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.241+0000 7f7e9b200640 1 -- 172.21.15.49:0/2753256290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7e7806c8f0 msgr2=0x7f7e7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.241+0000 7f7e9b200640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7e7806c8f0 0x7f7e7806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0098f0 tx=0x7f7e8c008040 comp rx=0 tx=0).stop 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.241+0000 7f7e9b200640 1 -- 172.21.15.49:0/2753256290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c15b410 msgr2=0x7f7e9c1f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.241+0000 7f7e9b200640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c15b410 0x7f7e9c1f4390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7e8402e100 tx=0x7f7e840046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.242+0000 7f7e9b200640 1 -- 172.21.15.49:0/2753256290 shutdown_connections 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.242+0000 7f7e9b200640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7e7806c8f0 0x7f7e7806edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.242+0000 7f7e9b200640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e9c15f110 0x7f7e9c15dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.242+0000 7f7e9b200640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c15b410 0x7f7e9c1f4390 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.242+0000 7f7e9b200640 1 --2- 172.21.15.49:0/2753256290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e9c0cf530 0x7f7e9c141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.242+0000 7f7e9b200640 1 -- 172.21.15.49:0/2753256290 >> 172.21.15.49:0/2753256290 conn(0x7f7e9c0b23a0 msgr2=0x7f7e9c06dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.242+0000 7f7e9b200640 1 -- 172.21.15.49:0/2753256290 shutdown_connections 2024-04-15T15:19:48.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.242+0000 7f7e9b200640 1 -- 172.21.15.49:0/2753256290 wait complete. 2024-04-15T15:19:48.267 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-15T15:19:48.282 INFO:tasks.ceph.mds.d.smithi107.stderr: -8629> 2024-04-15T15:18:48.961+0000 7ffa10c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:19:48.358 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-15T15:19:48.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.492+0000 7f2583200640 1 Processor -- start 2024-04-15T15:19:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.492+0000 7f2583200640 1 -- start start 2024-04-15T15:19:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.494+0000 7f2583200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f257c15a440 0x7f257c15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.494+0000 7f2583200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f257c06fc80 0x7f257c0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.494+0000 7f2583200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f257c15d520 con 0x7f257c0cd620 2024-04-15T15:19:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.494+0000 7f2583200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f257c06e280 con 0x7f257c15a440 2024-04-15T15:19:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.494+0000 7f2583200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f257c15d800 con 0x7f257c06fc80 2024-04-15T15:19:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.494+0000 7f257be00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f257c15a440 0x7f257c15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.494+0000 7f257be00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f257c15a440 0x7f257c15cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56354/0 (socket says 172.21.15.49:56354) 2024-04-15T15:19:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.494+0000 7f257be00640 1 -- 172.21.15.49:0/3463053592 learned_addr learned my addr 172.21.15.49:0/3463053592 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.495+0000 7f257b400640 1 --2- 172.21.15.49:0/3463053592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f257c06fc80 0x7f257c0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.495+0000 7f257be00640 1 -- 172.21.15.49:0/3463053592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f257c06fc80 msgr2=0x7f257c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.495+0000 7f257be00640 1 --2- 172.21.15.49:0/3463053592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f257c06fc80 0x7f257c0cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.495+0000 7f257be00640 1 -- 172.21.15.49:0/3463053592 >> v1:172.21.15.107:6789/0 conn(0x7f257c0cd620 legacy=0x7f257c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.495+0000 7f257be00640 1 -- 172.21.15.49:0/3463053592 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c15dae0 con 0x7f257c15a440 2024-04-15T15:19:48.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.497+0000 7f257be00640 1 --2- 172.21.15.49:0/3463053592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 0x7f257c15cef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f256c009870 tx=0x7f256c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58876eb8e504e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.497+0000 7f257aa00640 1 -- 172.21.15.49:0/3463053592 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c034070 con 0x7f257c15a440 2024-04-15T15:19:48.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.497+0000 7f257aa00640 1 -- 172.21.15.49:0/3463053592 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256c030840 con 0x7f257c15a440 2024-04-15T15:19:48.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.497+0000 7f257aa00640 1 -- 172.21.15.49:0/3463053592 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c030c10 con 0x7f257c15a440 2024-04-15T15:19:48.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.498+0000 7f2583200640 1 -- 172.21.15.49:0/3463053592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 msgr2=0x7f257c15cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.498+0000 7f2583200640 1 --2- 172.21.15.49:0/3463053592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 0x7f257c15cef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f256c009870 tx=0x7f256c02b980 comp rx=0 tx=0).stop 2024-04-15T15:19:48.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.498+0000 7f2583200640 1 -- 172.21.15.49:0/3463053592 shutdown_connections 2024-04-15T15:19:48.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.498+0000 7f2583200640 1 --2- 172.21.15.49:0/3463053592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f257c06fc80 0x7f257c0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.498+0000 7f2583200640 1 --2- 172.21.15.49:0/3463053592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 0x7f257c15cef0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.498+0000 7f2583200640 1 -- 172.21.15.49:0/3463053592 >> 172.21.15.49:0/3463053592 conn(0x7f257c0b2460 msgr2=0x7f257c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:48.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.498+0000 7f2583200640 1 -- 172.21.15.49:0/3463053592 shutdown_connections 2024-04-15T15:19:48.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.498+0000 7f2583200640 1 -- 172.21.15.49:0/3463053592 wait complete. 2024-04-15T15:19:48.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.500+0000 7f2583200640 1 Processor -- start 2024-04-15T15:19:48.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.500+0000 7f2583200640 1 -- start start 2024-04-15T15:19:48.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f2583200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f257c06fc80 0x7f257c159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f2583200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c0cd620 0x7f257c0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f2583200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 0x7f257c0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f2583200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f257c06f510 con 0x7f257c15a440 2024-04-15T15:19:48.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f2583200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f257c15ddc0 con 0x7f257c0cd620 2024-04-15T15:19:48.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f2583200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f257c072860 con 0x7f257c06fc80 2024-04-15T15:19:48.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f2581200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 0x7f257c0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f2581200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 0x7f257c0b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56356/0 (socket says 172.21.15.49:56356) 2024-04-15T15:19:48.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f2581200640 1 -- 172.21.15.49:0/299371804 learned_addr learned my addr 172.21.15.49:0/299371804 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:48.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f257be00640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f257c06fc80 0x7f257c159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.501+0000 7f257b400640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c0cd620 0x7f257c0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2581200640 1 -- 172.21.15.49:0/299371804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f257c06fc80 msgr2=0x7f257c159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2581200640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f257c06fc80 0x7f257c159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2581200640 1 -- 172.21.15.49:0/299371804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c0cd620 msgr2=0x7f257c0b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:48.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2581200640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c0cd620 0x7f257c0b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:48.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2581200640 1 -- 172.21.15.49:0/299371804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c0aef60 con 0x7f257c15a440 2024-04-15T15:19:48.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2581200640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 0x7f257c0b1a20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f257400c730 tx=0x7f257400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2573e00640 1 -- 172.21.15.49:0/299371804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574003b70 con 0x7f257c15a440 2024-04-15T15:19:48.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2573e00640 1 -- 172.21.15.49:0/299371804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2574003d10 con 0x7f257c15a440 2024-04-15T15:19:48.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2573e00640 1 -- 172.21.15.49:0/299371804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257400aa90 con 0x7f257c15a440 2024-04-15T15:19:48.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2583200640 1 -- 172.21.15.49:0/299371804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257c0af240 con 0x7f257c15a440 2024-04-15T15:19:48.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.502+0000 7f2583200640 1 -- 172.21.15.49:0/299371804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f257c159590 con 0x7f257c15a440 2024-04-15T15:19:48.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.504+0000 7f2583200640 1 -- 172.21.15.49:0/299371804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257c0af760 con 0x7f257c15a440 2024-04-15T15:19:48.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.505+0000 7f2573e00640 1 -- 172.21.15.49:0/299371804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f257400ac30 con 0x7f257c15a440 2024-04-15T15:19:48.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.505+0000 7f2573e00640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f255c06c8f0 0x7f255c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:48.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.506+0000 7f2573e00640 1 -- 172.21.15.49:0/299371804 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5911+0+0 (secure 0 0 0) 0x7f257408bbc0 con 0x7f257c15a440 2024-04-15T15:19:48.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.506+0000 7f257be00640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f255c06c8f0 0x7f255c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:48.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.506+0000 7f257be00640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f255c06c8f0 0x7f255c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f256c036000 tx=0x7f256c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:48.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.510+0000 7f2573e00640 1 -- 172.21.15.49:0/299371804 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2574059ec0 con 0x7f257c15a440 2024-04-15T15:19:48.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:48.756+0000 7f2583200640 1 -- 172.21.15.49:0/299371804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f257c068c80 con 0x7f257c15a440 2024-04-15T15:19:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.242+0000 7f2573e00640 1 -- 172.21.15.49:0/299371804 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f25740597c0 con 0x7f257c15a440 2024-04-15T15:19:49.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.245+0000 7f2583200640 1 -- 172.21.15.49:0/299371804 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f255c06c8f0 msgr2=0x7f255c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.245+0000 7f2583200640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f255c06c8f0 0x7f255c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f256c036000 tx=0x7f256c03b040 comp rx=0 tx=0).stop 2024-04-15T15:19:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.245+0000 7f2583200640 1 -- 172.21.15.49:0/299371804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 msgr2=0x7f257c0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.245+0000 7f2583200640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 0x7f257c0b1a20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f257400c730 tx=0x7f257400cbf0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.246+0000 7f2583200640 1 -- 172.21.15.49:0/299371804 shutdown_connections 2024-04-15T15:19:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.246+0000 7f2583200640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f255c06c8f0 0x7f255c06edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.246+0000 7f2583200640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c15a440 0x7f257c0b1a20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.246+0000 7f2583200640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c0cd620 0x7f257c0b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.246+0000 7f2583200640 1 --2- 172.21.15.49:0/299371804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f257c06fc80 0x7f257c159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.246+0000 7f2583200640 1 -- 172.21.15.49:0/299371804 >> 172.21.15.49:0/299371804 conn(0x7f257c0b2460 msgr2=0x7f257c15f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:49.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.246+0000 7f2583200640 1 -- 172.21.15.49:0/299371804 shutdown_connections 2024-04-15T15:19:49.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.246+0000 7f2583200640 1 -- 172.21.15.49:0/299371804 wait complete. 2024-04-15T15:19:49.274 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-15T15:19:49.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.508+0000 7f050ae00640 1 Processor -- start 2024-04-15T15:19:49.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.508+0000 7f050ae00640 1 -- start start 2024-04-15T15:19:49.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.510+0000 7f050ae00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05040cec60 0x7f0504071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.510+0000 7f050ae00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f050415d420 0x7f050406f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.510+0000 7f050ae00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f050413fc70 con 0x7f0504072450 2024-04-15T15:19:49.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.510+0000 7f050ae00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f050413f9e0 con 0x7f050415d420 2024-04-15T15:19:49.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.510+0000 7f050ae00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0504072890 con 0x7f05040cec60 2024-04-15T15:19:49.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.510+0000 7f050a400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f050415d420 0x7f050406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.510+0000 7f050a400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f050415d420 0x7f050406f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56368/0 (socket says 172.21.15.49:56368) 2024-04-15T15:19:49.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.510+0000 7f050a400640 1 -- 172.21.15.49:0/1618012959 learned_addr learned my addr 172.21.15.49:0/1618012959 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:49.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.511+0000 7f0509a00640 1 --2- 172.21.15.49:0/1618012959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05040cec60 0x7f0504071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.511+0000 7f050a400640 1 -- 172.21.15.49:0/1618012959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05040cec60 msgr2=0x7f0504071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:49.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.511+0000 7f050a400640 1 --2- 172.21.15.49:0/1618012959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05040cec60 0x7f0504071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.511+0000 7f050a400640 1 -- 172.21.15.49:0/1618012959 >> v1:172.21.15.107:6789/0 conn(0x7f0504072450 legacy=0x7f050415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:49.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.511+0000 7f050a400640 1 -- 172.21.15.49:0/1618012959 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0504141a00 con 0x7f050415d420 2024-04-15T15:19:49.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.511+0000 7f0509a00640 1 --2- 172.21.15.49:0/1618012959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05040cec60 0x7f0504071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:49.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.512+0000 7f050a400640 1 --2- 172.21.15.49:0/1618012959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f050415d420 0x7f050406f110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009e50 tx=0x7f04fc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb60898e9089b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.512+0000 7f0503e00640 1 -- 172.21.15.49:0/1618012959 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc02cb00 con 0x7f050415d420 2024-04-15T15:19:49.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.512+0000 7f0503e00640 1 -- 172.21.15.49:0/1618012959 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04fc02cca0 con 0x7f050415d420 2024-04-15T15:19:49.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.512+0000 7f0503e00640 1 -- 172.21.15.49:0/1618012959 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc034610 con 0x7f050415d420 2024-04-15T15:19:49.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.513+0000 7f050ae00640 1 -- 172.21.15.49:0/1618012959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f050415d420 msgr2=0x7f050406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.513+0000 7f050ae00640 1 --2- 172.21.15.49:0/1618012959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f050415d420 0x7f050406f110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009e50 tx=0x7f04fc02bf60 comp rx=0 tx=0).stop 2024-04-15T15:19:49.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.513+0000 7f050ae00640 1 -- 172.21.15.49:0/1618012959 shutdown_connections 2024-04-15T15:19:49.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.513+0000 7f050ae00640 1 --2- 172.21.15.49:0/1618012959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f050415d420 0x7f050406f110 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.513+0000 7f050ae00640 1 --2- 172.21.15.49:0/1618012959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05040cec60 0x7f0504071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.513+0000 7f050ae00640 1 -- 172.21.15.49:0/1618012959 >> 172.21.15.49:0/1618012959 conn(0x7f05040b6170 msgr2=0x7f050406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:49.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.513+0000 7f050ae00640 1 -- 172.21.15.49:0/1618012959 shutdown_connections 2024-04-15T15:19:49.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.513+0000 7f050ae00640 1 -- 172.21.15.49:0/1618012959 wait complete. 2024-04-15T15:19:49.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.515+0000 7f050ae00640 1 Processor -- start 2024-04-15T15:19:49.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.515+0000 7f050ae00640 1 -- start start 2024-04-15T15:19:49.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.516+0000 7f050ae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504072450 0x7f05041527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.516+0000 7f050ae00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05040cec60 0x7f0504155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.516+0000 7f050ae00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f050415d420 0x7f05041563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.516+0000 7f050ae00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f05041401f0 con 0x7f05040cec60 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.516+0000 7f050ae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f050413ff10 con 0x7f0504072450 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.516+0000 7f050ae00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f05041404d0 con 0x7f050415d420 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.516+0000 7f0509000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05040cec60 0x7f0504155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.517+0000 7f0509000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05040cec60 0x7f0504155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56382/0 (socket says 172.21.15.49:56382) 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.517+0000 7f0509000640 1 -- 172.21.15.49:0/1957854044 learned_addr learned my addr 172.21.15.49:0/1957854044 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.517+0000 7f0509a00640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504072450 0x7f05041527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.517+0000 7f050a400640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f050415d420 0x7f05041563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.517+0000 7f0509000640 1 -- 172.21.15.49:0/1957854044 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f050415d420 msgr2=0x7f05041563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.517+0000 7f0509000640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f050415d420 0x7f05041563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.517+0000 7f0509000640 1 -- 172.21.15.49:0/1957854044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504072450 msgr2=0x7f05041527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:49.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.517+0000 7f0509000640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504072450 0x7f05041527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:49.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.517+0000 7f0509000640 1 -- 172.21.15.49:0/1957854044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05041574d0 con 0x7f05040cec60 2024-04-15T15:19:49.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.517+0000 7f0509000640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05040cec60 0x7f0504155e80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b3f0 tx=0x7f04f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.518+0000 7f0501600640 1 -- 172.21.15.49:0/1957854044 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0015020 con 0x7f05040cec60 2024-04-15T15:19:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.518+0000 7f0501600640 1 -- 172.21.15.49:0/1957854044 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f0011830 con 0x7f05040cec60 2024-04-15T15:19:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.518+0000 7f0501600640 1 -- 172.21.15.49:0/1957854044 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0011c20 con 0x7f05040cec60 2024-04-15T15:19:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.518+0000 7f050ae00640 1 -- 172.21.15.49:0/1957854044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05041577b0 con 0x7f05040cec60 2024-04-15T15:19:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.518+0000 7f050ae00640 1 -- 172.21.15.49:0/1957854044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0504157d70 con 0x7f05040cec60 2024-04-15T15:19:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.519+0000 7f050ae00640 1 -- 172.21.15.49:0/1957854044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0504157260 con 0x7f05040cec60 2024-04-15T15:19:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.520+0000 7f0501600640 1 -- 172.21.15.49:0/1957854044 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04f0018020 con 0x7f05040cec60 2024-04-15T15:19:49.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.521+0000 7f0501600640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f04e406c8f0 0x7f04e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:49.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.521+0000 7f0501600640 1 -- 172.21.15.49:0/1957854044 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5911+0+0 (secure 0 0 0) 0x7f04f008b990 con 0x7f05040cec60 2024-04-15T15:19:49.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.521+0000 7f0509a00640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f04e406c8f0 0x7f04e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:49.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.522+0000 7f0509a00640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f04e406c8f0 0x7f04e406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04f800a990 tx=0x7f04f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:49.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.526+0000 7f0501600640 1 -- 172.21.15.49:0/1957854044 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04f001f030 con 0x7f05040cec60 2024-04-15T15:19:49.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:49.731+0000 7f050ae00640 1 -- 172.21.15.49:0/1957854044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f05040730e0 con 0x7f05040cec60 2024-04-15T15:19:50.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.253+0000 7f0501600640 1 -- 172.21.15.49:0/1957854044 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7f04f00172c0 con 0x7f05040cec60 2024-04-15T15:19:50.257 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2024-04-15T15:19:50.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.258+0000 7f050ae00640 1 -- 172.21.15.49:0/1957854044 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f04e406c8f0 msgr2=0x7f04e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.258+0000 7f050ae00640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f04e406c8f0 0x7f04e406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04f800a990 tx=0x7f04f8008040 comp rx=0 tx=0).stop 2024-04-15T15:19:50.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.258+0000 7f050ae00640 1 -- 172.21.15.49:0/1957854044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05040cec60 msgr2=0x7f0504155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.258+0000 7f050ae00640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05040cec60 0x7f0504155e80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b3f0 tx=0x7f04f000b8b0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.259+0000 7f050ae00640 1 -- 172.21.15.49:0/1957854044 shutdown_connections 2024-04-15T15:19:50.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.259+0000 7f050ae00640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f04e406c8f0 0x7f04e406edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.259+0000 7f050ae00640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f050415d420 0x7f05041563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.259+0000 7f050ae00640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05040cec60 0x7f0504155e80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.259+0000 7f050ae00640 1 --2- 172.21.15.49:0/1957854044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504072450 0x7f05041527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.259+0000 7f050ae00640 1 -- 172.21.15.49:0/1957854044 >> 172.21.15.49:0/1957854044 conn(0x7f05040b6170 msgr2=0x7f050415fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:50.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.259+0000 7f050ae00640 1 -- 172.21.15.49:0/1957854044 shutdown_connections 2024-04-15T15:19:50.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.259+0000 7f050ae00640 1 -- 172.21.15.49:0/1957854044 wait complete. 2024-04-15T15:19:50.290 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-15T15:19:50.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.502+0000 7fcbd9600640 1 Processor -- start 2024-04-15T15:19:50.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.502+0000 7fcbd9600640 1 -- start start 2024-04-15T15:19:50.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.504+0000 7fcbd9600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbd40cec60 0x7fcbd4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.504+0000 7fcbd9600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcbd4072450 0x7fcbd415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.504+0000 7fcbd9600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd413fc70 con 0x7fcbd415d250 2024-04-15T15:19:50.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.504+0000 7fcbd9600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fcbd413f9e0 con 0x7fcbd4072450 2024-04-15T15:19:50.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.504+0000 7fcbd9600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd4072890 con 0x7fcbd40cec60 2024-04-15T15:19:50.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.504+0000 7fcbd3400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcbd4072450 0x7fcbd415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.504+0000 7fcbd3400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcbd4072450 0x7fcbd415cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56398/0 (socket says 172.21.15.49:56398) 2024-04-15T15:19:50.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.504+0000 7fcbd3400640 1 -- 172.21.15.49:0/3109648034 learned_addr learned my addr 172.21.15.49:0/3109648034 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:50.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.505+0000 7fcbd3400640 1 -- 172.21.15.49:0/3109648034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbd40cec60 msgr2=0x7fcbd4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:50.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.505+0000 7fcbd3400640 1 --2- 172.21.15.49:0/3109648034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbd40cec60 0x7fcbd4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.505+0000 7fcbd3400640 1 -- 172.21.15.49:0/3109648034 >> v1:172.21.15.107:6789/0 conn(0x7fcbd415d250 legacy=0x7fcbd406f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:50.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.505+0000 7fcbd3400640 1 -- 172.21.15.49:0/3109648034 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd4141a00 con 0x7fcbd4072450 2024-04-15T15:19:50.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.506+0000 7fcbd3400640 1 --2- 172.21.15.49:0/3109648034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 0x7fcbd415cd10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009870 tx=0x7fcbbc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec244a11d4686390 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.506+0000 7fcbd2a00640 1 -- 172.21.15.49:0/3109648034 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc034070 con 0x7fcbd4072450 2024-04-15T15:19:50.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.506+0000 7fcbd2a00640 1 -- 172.21.15.49:0/3109648034 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbbc030840 con 0x7fcbd4072450 2024-04-15T15:19:50.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.506+0000 7fcbd2a00640 1 -- 172.21.15.49:0/3109648034 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc030c10 con 0x7fcbd4072450 2024-04-15T15:19:50.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.507+0000 7fcbd9600640 1 -- 172.21.15.49:0/3109648034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 msgr2=0x7fcbd415cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.507+0000 7fcbd9600640 1 --2- 172.21.15.49:0/3109648034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 0x7fcbd415cd10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009870 tx=0x7fcbbc02b980 comp rx=0 tx=0).stop 2024-04-15T15:19:50.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.507+0000 7fcbd9600640 1 -- 172.21.15.49:0/3109648034 shutdown_connections 2024-04-15T15:19:50.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.507+0000 7fcbd9600640 1 --2- 172.21.15.49:0/3109648034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 0x7fcbd415cd10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.507+0000 7fcbd9600640 1 --2- 172.21.15.49:0/3109648034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbd40cec60 0x7fcbd4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.507+0000 7fcbd9600640 1 -- 172.21.15.49:0/3109648034 >> 172.21.15.49:0/3109648034 conn(0x7fcbd40b6170 msgr2=0x7fcbd406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:50.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.507+0000 7fcbd9600640 1 -- 172.21.15.49:0/3109648034 shutdown_connections 2024-04-15T15:19:50.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.507+0000 7fcbd9600640 1 -- 172.21.15.49:0/3109648034 wait complete. 2024-04-15T15:19:50.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.509+0000 7fcbd9600640 1 Processor -- start 2024-04-15T15:19:50.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.509+0000 7fcbd9600640 1 -- start start 2024-04-15T15:19:50.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.510+0000 7fcbd9600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 0x7fcbd40b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.510+0000 7fcbd9600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbd40cec60 0x7fcbd40b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.510+0000 7fcbd9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbd415d250 0x7fcbd40b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.510+0000 7fcbd9600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd41401f0 con 0x7fcbd4072450 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.510+0000 7fcbd9600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd413ff10 con 0x7fcbd415d250 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.510+0000 7fcbd9600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd41404d0 con 0x7fcbd40cec60 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.510+0000 7fcbd3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 0x7fcbd40b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.510+0000 7fcbd3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 0x7fcbd40b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56406/0 (socket says 172.21.15.49:56406) 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.510+0000 7fcbd3e00640 1 -- 172.21.15.49:0/2017822767 learned_addr learned my addr 172.21.15.49:0/2017822767 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.511+0000 7fcbd3400640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbd40cec60 0x7fcbd40b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.511+0000 7fcbd8c00640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbd415d250 0x7fcbd40b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.511+0000 7fcbd3e00640 1 -- 172.21.15.49:0/2017822767 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbd40cec60 msgr2=0x7fcbd40b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.511+0000 7fcbd3e00640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbd40cec60 0x7fcbd40b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.511+0000 7fcbd3e00640 1 -- 172.21.15.49:0/2017822767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbd415d250 msgr2=0x7fcbd40b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:50.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.511+0000 7fcbd3e00640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbd415d250 0x7fcbd40b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:50.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.511+0000 7fcbd3e00640 1 -- 172.21.15.49:0/2017822767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd4154910 con 0x7fcbd4072450 2024-04-15T15:19:50.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.511+0000 7fcbd3e00640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 0x7fcbd40b1650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400ba80 tx=0x7fcbc400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.512+0000 7fcbcbe00640 1 -- 172.21.15.49:0/2017822767 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4019070 con 0x7fcbd4072450 2024-04-15T15:19:50.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.512+0000 7fcbcbe00640 1 -- 172.21.15.49:0/2017822767 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc400cc20 con 0x7fcbd4072450 2024-04-15T15:19:50.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.512+0000 7fcbcbe00640 1 -- 172.21.15.49:0/2017822767 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4014440 con 0x7fcbd4072450 2024-04-15T15:19:50.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.512+0000 7fcbd9600640 1 -- 172.21.15.49:0/2017822767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd40b1b90 con 0x7fcbd4072450 2024-04-15T15:19:50.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.512+0000 7fcbd9600640 1 -- 172.21.15.49:0/2017822767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd40ae9f0 con 0x7fcbd4072450 2024-04-15T15:19:50.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.513+0000 7fcbd9600640 1 -- 172.21.15.49:0/2017822767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd40b1dc0 con 0x7fcbd4072450 2024-04-15T15:19:50.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.514+0000 7fcbcbe00640 1 -- 172.21.15.49:0/2017822767 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbc401c070 con 0x7fcbd4072450 2024-04-15T15:19:50.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.515+0000 7fcbcbe00640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcbb006c8f0 0x7fcbb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:50.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.515+0000 7fcbcbe00640 1 -- 172.21.15.49:0/2017822767 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5473+0+0 (secure 0 0 0) 0x7fcbc408b760 con 0x7fcbd4072450 2024-04-15T15:19:50.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.515+0000 7fcbd3400640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcbb006c8f0 0x7fcbb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:50.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.516+0000 7fcbd3400640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcbb006c8f0 0x7fcbb006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc036000 tx=0x7fcbbc03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:50.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.520+0000 7fcbcbe00640 1 -- 172.21.15.49:0/2017822767 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbc4059580 con 0x7fcbd4072450 2024-04-15T15:19:50.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:50.729+0000 7fcbd9600640 1 -- 172.21.15.49:0/2017822767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcbd40730e0 con 0x7fcbd4072450 2024-04-15T15:19:51.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.264+0000 7fcbcbe00640 1 -- 172.21.15.49:0/2017822767 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7fcbc405d880 con 0x7fcbd4072450 2024-04-15T15:19:51.268 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2024-04-15T15:19:51.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.269+0000 7fcbd9600640 1 -- 172.21.15.49:0/2017822767 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcbb006c8f0 msgr2=0x7fcbb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.269+0000 7fcbd9600640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcbb006c8f0 0x7fcbb006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc036000 tx=0x7fcbbc03b040 comp rx=0 tx=0).stop 2024-04-15T15:19:51.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.269+0000 7fcbd9600640 1 -- 172.21.15.49:0/2017822767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 msgr2=0x7fcbd40b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.269+0000 7fcbd9600640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 0x7fcbd40b1650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400ba80 tx=0x7fcbc400bf40 comp rx=0 tx=0).stop 2024-04-15T15:19:51.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.270+0000 7fcbd9600640 1 -- 172.21.15.49:0/2017822767 shutdown_connections 2024-04-15T15:19:51.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.270+0000 7fcbd9600640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcbb006c8f0 0x7fcbb006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.270+0000 7fcbd9600640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbd40cec60 0x7fcbd40b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.270+0000 7fcbd9600640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbd415d250 0x7fcbd40b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.270+0000 7fcbd9600640 1 --2- 172.21.15.49:0/2017822767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd4072450 0x7fcbd40b1650 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.270+0000 7fcbd9600640 1 -- 172.21.15.49:0/2017822767 >> 172.21.15.49:0/2017822767 conn(0x7fcbd40b6170 msgr2=0x7fcbd406de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:51.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.271+0000 7fcbd9600640 1 -- 172.21.15.49:0/2017822767 shutdown_connections 2024-04-15T15:19:51.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.271+0000 7fcbd9600640 1 -- 172.21.15.49:0/2017822767 wait complete. 2024-04-15T15:19:51.300 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-15T15:19:51.301 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-15T15:19:51.301 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-15T15:19:51.301 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:19:51.301 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-15T15:19:51.301 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-15T15:19:51.301 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-15T15:19:51.301 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:19:51.302 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-15T15:19:51.302 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-15T15:19:51.302 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-15T15:19:51.302 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:19:51.302 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-15T15:19:51.302 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-15T15:19:51.302 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-15T15:19:51.302 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:19:51.321 INFO:tasks.ceph.mds.a:Stopped 2024-04-15T15:19:51.322 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-15T15:19:51.325 INFO:tasks.ceph.mds.b:Stopped 2024-04-15T15:19:51.325 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-15T15:19:51.328 INFO:tasks.ceph.mds.c:Stopped 2024-04-15T15:19:51.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-15T15:19:51.329 INFO:tasks.ceph.mds.d:Stopped 2024-04-15T15:19:51.329 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-15T15:19:51.374 INFO:tasks.ceph.mds.a:Started 2024-04-15T15:19:51.375 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:51.377 INFO:tasks.ceph.mds.b:Started 2024-04-15T15:19:51.377 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:51.381 INFO:tasks.ceph.mds.c:Started 2024-04-15T15:19:51.382 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:51.387 INFO:tasks.ceph.mds.d:Started 2024-04-15T15:19:51.387 DEBUG:teuthology.parallel:result is None 2024-04-15T15:19:51.387 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-15T15:19:51.540 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-15T15:19:51.544 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-15T15:19:51.554 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2024-04-15T15:19:51.557 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2024-04-15T15:19:51.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.597+0000 7f3249400640 1 Processor -- start 2024-04-15T15:19:51.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.597+0000 7f3249400640 1 -- start start 2024-04-15T15:19:51.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.598+0000 7f3249400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3244072100 0x7f3244072580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.598+0000 7f3249400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f324406f3f0 0x7f324406f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.598+0000 7f3249400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f324415d200 con 0x7f324413f6a0 2024-04-15T15:19:51.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.598+0000 7f3249400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f32440a95b0 con 0x7f324406f3f0 2024-04-15T15:19:51.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.598+0000 7f3249400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f324415d4e0 con 0x7f3244072100 2024-04-15T15:19:51.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.599+0000 7f3243200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f324406f3f0 0x7f324406f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.599+0000 7f3243200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f324406f3f0 0x7f324406f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:59554/0 (socket says 172.21.15.49:59554) 2024-04-15T15:19:51.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.599+0000 7f3243200640 1 -- 172.21.15.49:0/3193371068 learned_addr learned my addr 172.21.15.49:0/3193371068 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:51.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.599+0000 7f3242800640 1 --1- 172.21.15.49:0/3193371068 >> v1:172.21.15.107:6789/0 conn(0x7f324413f6a0 0x7f32440ceda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:51936/0 (socket says 172.21.15.49:51936) 2024-04-15T15:19:51.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.599+0000 7f3241e00640 1 --2- 172.21.15.49:0/3193371068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3244072100 0x7f3244072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.599+0000 7f3243200640 1 -- 172.21.15.49:0/3193371068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3244072100 msgr2=0x7f3244072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:51.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.599+0000 7f3243200640 1 --2- 172.21.15.49:0/3193371068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3244072100 0x7f3244072580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.599+0000 7f3243200640 1 -- 172.21.15.49:0/3193371068 >> v1:172.21.15.107:6789/0 conn(0x7f324413f6a0 legacy=0x7f32440ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.599+0000 7f3243200640 1 -- 172.21.15.49:0/3193371068 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324415d7c0 con 0x7f324406f3f0 2024-04-15T15:19:51.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.600+0000 7f3243200640 1 --2- 172.21.15.49:0/3193371068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f324406f3f0 0x7f324406f860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f323c009e50 tx=0x7f323c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e34ad2e4fd8fe07a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.601+0000 7f3241400640 1 -- 172.21.15.49:0/3193371068 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c02cb00 con 0x7f324406f3f0 2024-04-15T15:19:51.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.601+0000 7f3241400640 1 -- 172.21.15.49:0/3193371068 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c02cca0 con 0x7f324406f3f0 2024-04-15T15:19:51.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.601+0000 7f3241400640 1 -- 172.21.15.49:0/3193371068 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c034610 con 0x7f324406f3f0 2024-04-15T15:19:51.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.601+0000 7f3249400640 1 -- 172.21.15.49:0/3193371068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f324406f3f0 msgr2=0x7f324406f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.601+0000 7f3249400640 1 --2- 172.21.15.49:0/3193371068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f324406f3f0 0x7f324406f860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f323c009e50 tx=0x7f323c02bf60 comp rx=0 tx=0).stop 2024-04-15T15:19:51.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.601+0000 7f3249400640 1 -- 172.21.15.49:0/3193371068 shutdown_connections 2024-04-15T15:19:51.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.601+0000 7f3249400640 1 --2- 172.21.15.49:0/3193371068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f324406f3f0 0x7f324406f860 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.601+0000 7f3249400640 1 --2- 172.21.15.49:0/3193371068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3244072100 0x7f3244072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.601+0000 7f3249400640 1 -- 172.21.15.49:0/3193371068 >> 172.21.15.49:0/3193371068 conn(0x7f32440b2460 msgr2=0x7f32440b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:51.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.602+0000 7f3249400640 1 -- 172.21.15.49:0/3193371068 shutdown_connections 2024-04-15T15:19:51.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.602+0000 7f3249400640 1 -- 172.21.15.49:0/3193371068 wait complete. 2024-04-15T15:19:51.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.603+0000 7f3249400640 1 Processor -- start 2024-04-15T15:19:51.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.603+0000 7f3249400640 1 -- start start 2024-04-15T15:19:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.604+0000 7f3249400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f324406f3f0 0x7f324413bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.604+0000 7f3249400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3244072100 0x7f3244158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.604+0000 7f3249400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f324413f6a0 0x7f3244159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.604+0000 7f3249400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f324415dd80 con 0x7f3244072100 2024-04-15T15:19:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.604+0000 7f3249400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f324413fef0 con 0x7f324406f3f0 2024-04-15T15:19:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.604+0000 7f3249400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f324415ce60 con 0x7f324413f6a0 2024-04-15T15:19:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3241e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3244072100 0x7f3244158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3241e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3244072100 0x7f3244158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56452/0 (socket says 172.21.15.49:56452) 2024-04-15T15:19:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3241e00640 1 -- 172.21.15.49:0/2985770070 learned_addr learned my addr 172.21.15.49:0/2985770070 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3242800640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f324406f3f0 0x7f324413bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3243200640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f324413f6a0 0x7f3244159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3241e00640 1 -- 172.21.15.49:0/2985770070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f324413f6a0 msgr2=0x7f3244159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3241e00640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f324413f6a0 0x7f3244159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3241e00640 1 -- 172.21.15.49:0/2985770070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f324406f3f0 msgr2=0x7f324413bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3241e00640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f324406f3f0 0x7f324413bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3241e00640 1 -- 172.21.15.49:0/2985770070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32441579e0 con 0x7f3244072100 2024-04-15T15:19:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.605+0000 7f3242800640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f324406f3f0 0x7f324413bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:51.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.606+0000 7f3241e00640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3244072100 0x7f3244158a80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f323000e9b0 tx=0x7f323000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.606+0000 7f323aa00640 1 -- 172.21.15.49:0/2985770070 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230017070 con 0x7f3244072100 2024-04-15T15:19:51.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.606+0000 7f323aa00640 1 -- 172.21.15.49:0/2985770070 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323000c9a0 con 0x7f3244072100 2024-04-15T15:19:51.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.606+0000 7f323aa00640 1 -- 172.21.15.49:0/2985770070 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323000cd90 con 0x7f3244072100 2024-04-15T15:19:51.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.606+0000 7f3249400640 1 -- 172.21.15.49:0/2985770070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3244157cc0 con 0x7f3244072100 2024-04-15T15:19:51.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.606+0000 7f3249400640 1 -- 172.21.15.49:0/2985770070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3244158280 con 0x7f3244072100 2024-04-15T15:19:51.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.607+0000 7f3249400640 1 -- 172.21.15.49:0/2985770070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3244068c80 con 0x7f3244072100 2024-04-15T15:19:51.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.608+0000 7f323aa00640 1 -- 172.21.15.49:0/2985770070 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f323001a070 con 0x7f3244072100 2024-04-15T15:19:51.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.609+0000 7f323aa00640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f322406c8f0 0x7f322406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:51.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.609+0000 7f323aa00640 1 -- 172.21.15.49:0/2985770070 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4943+0+0 (secure 0 0 0) 0x7f323008bcc0 con 0x7f3244072100 2024-04-15T15:19:51.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.609+0000 7f3242800640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f322406c8f0 0x7f322406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:51.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.610+0000 7f3242800640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f322406c8f0 0x7f322406edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3234007fd0 tx=0x7f3234007d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:51.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.614+0000 7f323aa00640 1 -- 172.21.15.49:0/2985770070 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3230092080 con 0x7f3244072100 2024-04-15T15:19:51.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:51.831+0000 7f3249400640 1 -- 172.21.15.49:0/2985770070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3244072ba0 con 0x7f3244072100 2024-04-15T15:19:52.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.288+0000 7f323aa00640 1 -- 172.21.15.49:0/2985770070 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) ==== 97+0+0 (secure 0 0 0) 0x7f323005a350 con 0x7f3244072100 2024-04-15T15:19:52.292 INFO:teuthology.orchestra.run.smithi049.stderr: removed all blocklist entries 2024-04-15T15:19:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 -- 172.21.15.49:0/2985770070 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f322406c8f0 msgr2=0x7f322406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f322406c8f0 0x7f322406edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3234007fd0 tx=0x7f3234007d60 comp rx=0 tx=0).stop 2024-04-15T15:19:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 -- 172.21.15.49:0/2985770070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3244072100 msgr2=0x7f3244158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3244072100 0x7f3244158a80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f323000e9b0 tx=0x7f323000ee70 comp rx=0 tx=0).stop 2024-04-15T15:19:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 -- 172.21.15.49:0/2985770070 shutdown_connections 2024-04-15T15:19:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f322406c8f0 0x7f322406edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f324413f6a0 0x7f3244159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3244072100 0x7f3244158a80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 --2- 172.21.15.49:0/2985770070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f324406f3f0 0x7f324413bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 -- 172.21.15.49:0/2985770070 >> 172.21.15.49:0/2985770070 conn(0x7f32440b2460 msgr2=0x7f324415b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:52.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 -- 172.21.15.49:0/2985770070 shutdown_connections 2024-04-15T15:19:52.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.292+0000 7f3249400640 1 -- 172.21.15.49:0/2985770070 wait complete. 2024-04-15T15:19:52.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-15T15:19:52.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.534+0000 7fe284e00640 1 Processor -- start 2024-04-15T15:19:52.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.535+0000 7fe284e00640 1 -- start start 2024-04-15T15:19:52.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.536+0000 7fe284e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe2800cf110 0x7fe280162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.536+0000 7fe284e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2800d0090 con 0x7fe280162020 2024-04-15T15:19:52.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.536+0000 7fe284e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2800cfad0 con 0x7fe2800d1c90 2024-04-15T15:19:52.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.536+0000 7fe284e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fe280163440 con 0x7fe2800cf110 2024-04-15T15:19:52.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.536+0000 7fe27f400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe280162020 0x7fe2800d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47966/0 (socket says 172.21.15.49:47966) 2024-04-15T15:19:52.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.536+0000 7fe27f400640 1 -- 172.21.15.49:0/2190199401 learned_addr learned my addr 172.21.15.49:0/2190199401 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2374481428 0 0) 0x7fe270001110 con 0x7fe280162020 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe264003520 con 0x7fe280162020 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27fe00640 1 --2- 172.21.15.49:0/2190199401 >> v2:172.21.15.107:3300/0 conn(0x7fe2800cf110 0x7fe280162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 57912144 0 0) 0x7fe264003520 con 0x7fe280162020 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 >> v2:172.21.15.107:3300/0 conn(0x7fe2800cf110 msgr2=0x7fe280162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 --2- 172.21.15.49:0/2190199401 >> v2:172.21.15.107:3300/0 conn(0x7fe2800cf110 0x7fe280162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 >> v1:172.21.15.107:6789/0 conn(0x7fe2800d1c90 legacy=0x7fe2800d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2800cfdb0 con 0x7fe280162020 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fe270001840 con 0x7fe280162020 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 >> v1:172.21.15.49:6789/0 conn(0x7fe280162020 legacy=0x7fe2800d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264003a00 0x7fe264003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe264004800 0x7fe264004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe26400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe280163440 con 0x7fe264005210 2024-04-15T15:19:52.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe26400c140 con 0x7fe264003a00 2024-04-15T15:19:52.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.537+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe26400c340 con 0x7fe264004800 2024-04-15T15:19:52.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.538+0000 7fe27fe00640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe26400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.538+0000 7fe27f400640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264003a00 0x7fe264003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.538+0000 7fe277e00640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe264004800 0x7fe264004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.538+0000 7fe27fe00640 1 -- 172.21.15.49:0/2190199401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe264004800 msgr2=0x7fe264004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:52.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.538+0000 7fe27fe00640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe264004800 0x7fe264004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.538+0000 7fe27fe00640 1 -- 172.21.15.49:0/2190199401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264003a00 msgr2=0x7fe264003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:52.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.538+0000 7fe27fe00640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe264003a00 0x7fe264003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.538+0000 7fe27fe00640 1 -- 172.21.15.49:0/2190199401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe264003520 con 0x7fe264005210 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.539+0000 7fe27fe00640 1 -- 172.21.15.49:0/2190199401 reap_dead start 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.539+0000 7fe27fe00640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe26400b9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe28006c9a0 tx=0x7fe264005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136e5e30981a611a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.540+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278009be0 con 0x7fe264005210 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.540+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe27802cc70 con 0x7fe264005210 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.540+0000 7fe27ea00640 1 -- 172.21.15.49:0/2190199401 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278003bd0 con 0x7fe264005210 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.540+0000 7fe284e00640 1 -- 172.21.15.49:0/2190199401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 msgr2=0x7fe26400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.540+0000 7fe284e00640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe26400b9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe28006c9a0 tx=0x7fe264005830 comp rx=0 tx=0).stop 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.540+0000 7fe284e00640 1 -- 172.21.15.49:0/2190199401 shutdown_connections 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.540+0000 7fe284e00640 1 --2- 172.21.15.49:0/2190199401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe26400b9c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.540+0000 7fe284e00640 1 -- 172.21.15.49:0/2190199401 >> 172.21.15.49:0/2190199401 conn(0x7fe28006cf00 msgr2=0x7fe28006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.540+0000 7fe284e00640 1 -- 172.21.15.49:0/2190199401 shutdown_connections 2024-04-15T15:19:52.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.540+0000 7fe284e00640 1 -- 172.21.15.49:0/2190199401 wait complete. 2024-04-15T15:19:52.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.542+0000 7fe284e00640 1 Processor -- start 2024-04-15T15:19:52.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.542+0000 7fe284e00640 1 -- start start 2024-04-15T15:19:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.543+0000 7fe284e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe280150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.543+0000 7fe284e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe280160670 0x7fe2800c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.543+0000 7fe284e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe280160af0 0x7fe2800ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.543+0000 7fe284e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe2800cf880 con 0x7fe264005210 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.543+0000 7fe284e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2800b7b30 con 0x7fe280160af0 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.543+0000 7fe284e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe2800d2e70 con 0x7fe280160670 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.544+0000 7fe27f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe280150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.544+0000 7fe27f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe280150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56460/0 (socket says 172.21.15.49:56460) 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.544+0000 7fe27f400640 1 -- 172.21.15.49:0/1042601753 learned_addr learned my addr 172.21.15.49:0/1042601753 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.544+0000 7fe27fe00640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe280160af0 0x7fe2800ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.544+0000 7fe27f400640 1 -- 172.21.15.49:0/1042601753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe280160670 msgr2=0x7fe2800c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.544+0000 7fe27f400640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe280160670 0x7fe2800c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.544+0000 7fe27f400640 1 -- 172.21.15.49:0/1042601753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe280160af0 msgr2=0x7fe2800ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.544+0000 7fe27f400640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe280160af0 0x7fe2800ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.544+0000 7fe27f400640 1 -- 172.21.15.49:0/1042601753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe280150d70 con 0x7fe264005210 2024-04-15T15:19:52.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.544+0000 7fe27f400640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe280150830 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe27000e730 tx=0x7fe27000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.545+0000 7fe277400640 1 -- 172.21.15.49:0/1042601753 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27000ee40 con 0x7fe264005210 2024-04-15T15:19:52.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.545+0000 7fe277400640 1 -- 172.21.15.49:0/1042601753 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe270018070 con 0x7fe264005210 2024-04-15T15:19:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.545+0000 7fe277400640 1 -- 172.21.15.49:0/1042601753 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27000cab0 con 0x7fe264005210 2024-04-15T15:19:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.545+0000 7fe284e00640 1 -- 172.21.15.49:0/1042601753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28015ef80 con 0x7fe264005210 2024-04-15T15:19:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.545+0000 7fe284e00640 1 -- 172.21.15.49:0/1042601753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28015f500 con 0x7fe264005210 2024-04-15T15:19:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.546+0000 7fe284e00640 1 -- 172.21.15.49:0/1042601753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2800755c0 con 0x7fe264005210 2024-04-15T15:19:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.547+0000 7fe277400640 1 -- 172.21.15.49:0/1042601753 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe27000cc50 con 0x7fe264005210 2024-04-15T15:19:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.548+0000 7fe277400640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe25c06c8f0 0x7fe25c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.548+0000 7fe277400640 1 -- 172.21.15.49:0/1042601753 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fe27008b7c0 con 0x7fe264005210 2024-04-15T15:19:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.548+0000 7fe277e00640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe25c06c8f0 0x7fe25c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.549+0000 7fe277e00640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe25c06c8f0 0x7fe25c06edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe280158150 tx=0x7fe268009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.553+0000 7fe277400640 1 -- 172.21.15.49:0/1042601753 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2800d0090 con 0x7fe264005210 2024-04-15T15:19:52.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.837+0000 7fe284e00640 1 -- 172.21.15.49:0/1042601753 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe2800b7650 con 0x7fe264005210 2024-04-15T15:19:52.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.838+0000 7fe277400640 1 -- 172.21.15.49:0/1042601753 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fe270059ea0 con 0x7fe264005210 2024-04-15T15:19:52.841 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:52.841 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:19:52.841 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_dump": [ 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.a", 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAVRR1m5WHRFRAAJXBz+oU3IyMQu/G5dsvabw==", 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.b", 2024-04-15T15:19:52.842 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAVRR1m6EcuOhAAU3JqIQ65mBuJ8sG0DuIwog==", 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.c", 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAVRR1mMUwqKBAAiSQOWs9ymyVAUExfk9Qclw==", 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.843 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.d", 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAWRR1m1ynhDhAAHZ/86Gje4O++XTEh5HZV4g==", 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.0", 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAZRR1mt7wUBxAAXcBmE8sA/hGwdFvQ0HHMYA==", 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.1", 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAcRR1mUT4+FhAAmdPWZRMVi+/75nhZVtfT9Q==", 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.845 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.2", 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAfRR1mYBvAMhAAyvQdSMQ0az5vuMX/lgzIwg==", 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.3", 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAjRR1m7DcBBBAAfsRuWgRb5te0UjkDcd0IMQ==", 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.4", 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQApRR1mBz6AIBAAp73aMr+Lp1bAZyyqqBALGQ==", 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2024-04-15T15:19:52.847 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.5", 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAsRR1m5tIQLBAAOAo1pTWsKMwiMwBF06pbjQ==", 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.6", 2024-04-15T15:19:52.848 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAvRR1mIOcqOBAA4feE1ltGRQjt5h3NXU74Aw==", 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.7", 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAzRR1mAPgPGRAAD2tksIXFnQdbN65/qYe5+A==", 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2024-04-15T15:19:52.849 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.0", 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAWRR1mkYQSHRAA0LcNsz2EApVHL/mhDGmMVg==", 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.850 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.1", 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAWRR1mn65aNBAAn12myNVRPC9WwpTHEEhtdg==", 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.2", 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAWRR1mf7kDKRAACANrDIeJ2N6CqazlX4/DpQ==", 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2024-04-15T15:19:52.851 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.3", 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAXRR1mX7HdAhAAOxlAH/My0eFkx9W5ZrDW0A==", 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.admin", 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAURR1mI18EBBAAwo7a8T/IQkJB5WvYZUtCCg==", 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-mds", 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBBRR1mIcpbHhAATABB0vBc6Avjkzb11Bj5bA==", 2024-04-15T15:19:52.853 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-mds" 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-mgr", 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBBRR1mf+lbHhAA3F2UeZM5mT4PfB2/brlRjQ==", 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-osd", 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBBRR1mqAVcHhAA58Jg1BBpMRmKM4tE7++FeA==", 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-osd" 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-rbd", 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBBRR1mvSBcHhAAFv9jx14UG/soYznvuX1CYw==", 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBBRR1m8eJcHhAAzUb1CqbDZ060Qs2rQnR4JA==", 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-rgw", 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBBRR1mzwldHhAAL/WKn3usO1DmyHQtIx3hiQ==", 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.x", 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAVRR1mKUPhCBAA9NPB6Z8xXRqJgEqs49pi7A==", 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.y", 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAURR1mhIt2ORAAxgHgedKRzpFt/ARWPZjP3g==", 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-15T15:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.841+0000 7fe284e00640 1 -- 172.21.15.49:0/1042601753 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe25c06c8f0 msgr2=0x7fe25c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.841+0000 7fe284e00640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe25c06c8f0 0x7fe25c06edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe280158150 tx=0x7fe268009040 comp rx=0 tx=0).stop 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.841+0000 7fe284e00640 1 -- 172.21.15.49:0/1042601753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 msgr2=0x7fe280150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.841+0000 7fe284e00640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe280150830 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe27000e730 tx=0x7fe27000ebf0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.841+0000 7fe284e00640 1 -- 172.21.15.49:0/1042601753 shutdown_connections 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.841+0000 7fe284e00640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe25c06c8f0 0x7fe25c06edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.841+0000 7fe284e00640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe280160670 0x7fe2800c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.841+0000 7fe284e00640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe280160af0 0x7fe2800ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.841+0000 7fe284e00640 1 --2- 172.21.15.49:0/1042601753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264005210 0x7fe280150830 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.841+0000 7fe284e00640 1 -- 172.21.15.49:0/1042601753 >> 172.21.15.49:0/1042601753 conn(0x7fe28006cf00 msgr2=0x7fe2800ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.842+0000 7fe284e00640 1 -- 172.21.15.49:0/1042601753 shutdown_connections 2024-04-15T15:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:52.842+0000 7fe284e00640 1 -- 172.21.15.49:0/1042601753 wait complete. 2024-04-15T15:19:52.865 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-15T15:19:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.077+0000 7f2d32400640 1 Processor -- start 2024-04-15T15:19:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.078+0000 7f2d32400640 1 -- start start 2024-04-15T15:19:53.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.079+0000 7f2d32400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2d2c15eab0 0x7f2d2c162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.079+0000 7f2d32400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d2c162b70 0x7f2d2c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.079+0000 7f2d32400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2c0b78f0 con 0x7f2d2c0d1a90 2024-04-15T15:19:53.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.079+0000 7f2d32400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2c0d2e50 con 0x7f2d2c15eab0 2024-04-15T15:19:53.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.079+0000 7f2d32400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2c163010 con 0x7f2d2c162b70 2024-04-15T15:19:53.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.080+0000 7f2d31a00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2d2c0d1a90 0x7f2d2c0b6240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47970/0 (socket says 172.21.15.49:47970) 2024-04-15T15:19:53.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.080+0000 7f2d31a00640 1 -- 172.21.15.49:0/1937185584 learned_addr learned my addr 172.21.15.49:0/1937185584 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:53.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.080+0000 7f2d31000640 1 --2- 172.21.15.49:0/1937185584 >> v2:172.21.15.49:3300/0 conn(0x7f2d2c15eab0 0x7f2d2c162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.080+0000 7f2d2be00640 1 --2- 172.21.15.49:0/1937185584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d2c162b70 0x7f2d2c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.080+0000 7f2d2b400640 1 -- 172.21.15.49:0/1937185584 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1803826903 0 0) 0x7f2d24001490 con 0x7f2d2c0d1a90 2024-04-15T15:19:53.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.080+0000 7f2d2b400640 1 -- 172.21.15.49:0/1937185584 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d10003520 con 0x7f2d2c0d1a90 2024-04-15T15:19:53.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.080+0000 7f2d31000640 1 -- 172.21.15.49:0/1937185584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d2c162b70 msgr2=0x7f2d2c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:53.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.080+0000 7f2d31000640 1 --2- 172.21.15.49:0/1937185584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d2c162b70 0x7f2d2c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.080+0000 7f2d31000640 1 -- 172.21.15.49:0/1937185584 >> v1:172.21.15.49:6789/0 conn(0x7f2d2c0d1a90 legacy=0x7f2d2c0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.081+0000 7f2d31000640 1 -- 172.21.15.49:0/1937185584 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c1632f0 con 0x7f2d2c15eab0 2024-04-15T15:19:53.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.082+0000 7f2d31000640 1 --2- 172.21.15.49:0/1937185584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c15eab0 0x7f2d2c162630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0097a0 tx=0x7f2d1c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bb74da40fe0a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.082+0000 7f2d2b400640 1 -- 172.21.15.49:0/1937185584 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c035020 con 0x7f2d2c15eab0 2024-04-15T15:19:53.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.082+0000 7f2d2b400640 1 -- 172.21.15.49:0/1937185584 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1c031830 con 0x7f2d2c15eab0 2024-04-15T15:19:53.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.082+0000 7f2d2b400640 1 -- 172.21.15.49:0/1937185584 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c031c00 con 0x7f2d2c15eab0 2024-04-15T15:19:53.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.082+0000 7f2d32400640 1 -- 172.21.15.49:0/1937185584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c15eab0 msgr2=0x7f2d2c162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.082+0000 7f2d32400640 1 --2- 172.21.15.49:0/1937185584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c15eab0 0x7f2d2c162630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0097a0 tx=0x7f2d1c02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.083+0000 7f2d32400640 1 -- 172.21.15.49:0/1937185584 shutdown_connections 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.083+0000 7f2d32400640 1 --2- 172.21.15.49:0/1937185584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d2c162b70 0x7f2d2c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.083+0000 7f2d32400640 1 --2- 172.21.15.49:0/1937185584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c15eab0 0x7f2d2c162630 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.083+0000 7f2d32400640 1 -- 172.21.15.49:0/1937185584 >> 172.21.15.49:0/1937185584 conn(0x7f2d2c073240 msgr2=0x7f2d2c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.083+0000 7f2d32400640 1 -- 172.21.15.49:0/1937185584 shutdown_connections 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.083+0000 7f2d32400640 1 -- 172.21.15.49:0/1937185584 wait complete. 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.084+0000 7f2d32400640 1 Processor -- start 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.085+0000 7f2d32400640 1 -- start start 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.085+0000 7f2d32400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c0d1a90 0x7f2d2c160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d32400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d2c15eab0 0x7f2d2c1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d32400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c162b70 0x7f2d2c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d32400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c0ba590 con 0x7f2d2c162b70 2024-04-15T15:19:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d32400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c15ee90 con 0x7f2d2c0d1a90 2024-04-15T15:19:53.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d32400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2c0ba710 con 0x7f2d2c15eab0 2024-04-15T15:19:53.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d31a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c162b70 0x7f2d2c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d31a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c162b70 0x7f2d2c15d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56478/0 (socket says 172.21.15.49:56478) 2024-04-15T15:19:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d31a00640 1 -- 172.21.15.49:0/4030451440 learned_addr learned my addr 172.21.15.49:0/4030451440 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d31000640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c0d1a90 0x7f2d2c160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d31a00640 1 -- 172.21.15.49:0/4030451440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d2c15eab0 msgr2=0x7f2d2c1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d31a00640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d2c15eab0 0x7f2d2c1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d31a00640 1 -- 172.21.15.49:0/4030451440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c0d1a90 msgr2=0x7f2d2c160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d31a00640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c0d1a90 0x7f2d2c160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d31a00640 1 -- 172.21.15.49:0/4030451440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c15af20 con 0x7f2d2c162b70 2024-04-15T15:19:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.086+0000 7f2d31000640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c0d1a90 0x7f2d2c160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.087+0000 7f2d31a00640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c162b70 0x7f2d2c15d630 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400cb10 tx=0x7f2d2400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.087+0000 7f2d28c00640 1 -- 172.21.15.49:0/4030451440 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24017070 con 0x7f2d2c162b70 2024-04-15T15:19:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.087+0000 7f2d28c00640 1 -- 172.21.15.49:0/4030451440 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d2400ac70 con 0x7f2d2c162b70 2024-04-15T15:19:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.087+0000 7f2d28c00640 1 -- 172.21.15.49:0/4030451440 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24012640 con 0x7f2d2c162b70 2024-04-15T15:19:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.087+0000 7f2d32400640 1 -- 172.21.15.49:0/4030451440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c160bd0 con 0x7f2d2c162b70 2024-04-15T15:19:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.087+0000 7f2d32400640 1 -- 172.21.15.49:0/4030451440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c161190 con 0x7f2d2c162b70 2024-04-15T15:19:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.088+0000 7f2d32400640 1 -- 172.21.15.49:0/4030451440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2c0755c0 con 0x7f2d2c162b70 2024-04-15T15:19:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.089+0000 7f2d28c00640 1 -- 172.21.15.49:0/4030451440 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d2401a070 con 0x7f2d2c162b70 2024-04-15T15:19:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.090+0000 7f2d28c00640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c8f0 0x7f2d0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.090+0000 7f2d28c00640 1 -- 172.21.15.49:0/4030451440 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f2d2408b6d0 con 0x7f2d2c162b70 2024-04-15T15:19:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.090+0000 7f2d31000640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c8f0 0x7f2d0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.091+0000 7f2d31000640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c8f0 0x7f2d0806edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c036000 tx=0x7f2d1c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.094+0000 7f2d28c00640 1 -- 172.21.15.49:0/4030451440 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d2c0b78f0 con 0x7f2d2c162b70 2024-04-15T15:19:53.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.381+0000 7f2d32400640 1 -- 172.21.15.49:0/4030451440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f2d2c0b9770 con 0x7f2d2c162b70 2024-04-15T15:19:53.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.384+0000 7f2d28c00640 1 -- 172.21.15.49:0/4030451440 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f2d2400ae20 con 0x7f2d2c162b70 2024-04-15T15:19:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 -- 172.21.15.49:0/4030451440 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c8f0 msgr2=0x7f2d0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c8f0 0x7f2d0806edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c036000 tx=0x7f2d1c008030 comp rx=0 tx=0).stop 2024-04-15T15:19:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 -- 172.21.15.49:0/4030451440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c162b70 msgr2=0x7f2d2c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c162b70 0x7f2d2c15d630 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400cb10 tx=0x7f2d2400a060 comp rx=0 tx=0).stop 2024-04-15T15:19:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 -- 172.21.15.49:0/4030451440 shutdown_connections 2024-04-15T15:19:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f2d0806c8f0 0x7f2d0806edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d2c15eab0 0x7f2d2c1605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d2c162b70 0x7f2d2c15d630 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 --2- 172.21.15.49:0/4030451440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c0d1a90 0x7f2d2c160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 -- 172.21.15.49:0/4030451440 >> 172.21.15.49:0/4030451440 conn(0x7f2d2c073240 msgr2=0x7f2d2c0cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:53.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 -- 172.21.15.49:0/4030451440 shutdown_connections 2024-04-15T15:19:53.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.387+0000 7f2d32400640 1 -- 172.21.15.49:0/4030451440 wait complete. 2024-04-15T15:19:53.407 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-15T15:19:53.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.630+0000 7f25f3400640 1 Processor -- start 2024-04-15T15:19:53.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.630+0000 7f25f3400640 1 -- start start 2024-04-15T15:19:53.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.632+0000 7f25f3400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25ec0b43b0 0x7f25ec0aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.632+0000 7f25f3400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25ec0af1f0 0x7f25ec072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.632+0000 7f25f3400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ec15f6b0 con 0x7f25ec148110 2024-04-15T15:19:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.632+0000 7f25f3400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f25ec15ff20 con 0x7f25ec0af1f0 2024-04-15T15:19:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.632+0000 7f25f3400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f25ec15f960 con 0x7f25ec0b43b0 2024-04-15T15:19:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.632+0000 7f25f1400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25ec0af1f0 0x7f25ec072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.632+0000 7f25f1400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25ec0af1f0 0x7f25ec072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56484/0 (socket says 172.21.15.49:56484) 2024-04-15T15:19:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.632+0000 7f25f0a00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f25ec148110 0x7f25ec15efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47982/0 (socket says 172.21.15.49:47982) 2024-04-15T15:19:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.632+0000 7f25f1400640 1 -- 172.21.15.49:0/512474007 learned_addr learned my addr 172.21.15.49:0/512474007 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:53.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.632+0000 7f25ebe00640 1 --2- 172.21.15.49:0/512474007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25ec0b43b0 0x7f25ec0aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.633+0000 7f25f1400640 1 -- 172.21.15.49:0/512474007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25ec0b43b0 msgr2=0x7f25ec0aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:53.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.633+0000 7f25f1400640 1 --2- 172.21.15.49:0/512474007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25ec0b43b0 0x7f25ec0aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.633+0000 7f25f1400640 1 -- 172.21.15.49:0/512474007 >> v1:172.21.15.49:6789/0 conn(0x7f25ec148110 legacy=0x7f25ec15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.633+0000 7f25f1400640 1 -- 172.21.15.49:0/512474007 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec15fc40 con 0x7f25ec0af1f0 2024-04-15T15:19:53.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.634+0000 7f25f1400640 1 --2- 172.21.15.49:0/512474007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0af1f0 0x7f25ec072100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009e50 tx=0x7f25e402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b840543c2e8e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.634+0000 7f25eb400640 1 -- 172.21.15.49:0/512474007 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e402cb00 con 0x7f25ec0af1f0 2024-04-15T15:19:53.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.634+0000 7f25eb400640 1 -- 172.21.15.49:0/512474007 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e402cca0 con 0x7f25ec0af1f0 2024-04-15T15:19:53.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.634+0000 7f25eb400640 1 -- 172.21.15.49:0/512474007 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4034610 con 0x7f25ec0af1f0 2024-04-15T15:19:53.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.635+0000 7f25f3400640 1 -- 172.21.15.49:0/512474007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0af1f0 msgr2=0x7f25ec072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.635+0000 7f25f3400640 1 --2- 172.21.15.49:0/512474007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0af1f0 0x7f25ec072100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009e50 tx=0x7f25e402bf60 comp rx=0 tx=0).stop 2024-04-15T15:19:53.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.635+0000 7f25f3400640 1 -- 172.21.15.49:0/512474007 shutdown_connections 2024-04-15T15:19:53.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.635+0000 7f25f3400640 1 --2- 172.21.15.49:0/512474007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0af1f0 0x7f25ec072100 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.635+0000 7f25f3400640 1 --2- 172.21.15.49:0/512474007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25ec0b43b0 0x7f25ec0aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.635+0000 7f25f3400640 1 -- 172.21.15.49:0/512474007 >> 172.21.15.49:0/512474007 conn(0x7f25ec0b2460 msgr2=0x7f25ec0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:53.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.635+0000 7f25f3400640 1 -- 172.21.15.49:0/512474007 shutdown_connections 2024-04-15T15:19:53.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.635+0000 7f25f3400640 1 -- 172.21.15.49:0/512474007 wait complete. 2024-04-15T15:19:53.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.637+0000 7f25f3400640 1 Processor -- start 2024-04-15T15:19:53.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.637+0000 7f25f3400640 1 -- start start 2024-04-15T15:19:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.638+0000 7f25f3400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25ec0af1f0 0x7f25ec146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.638+0000 7f25f3400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0b43b0 0x7f25ec0cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.638+0000 7f25f3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25ec148110 0x7f25ec13edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.638+0000 7f25f3400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1484f0 con 0x7f25ec0b43b0 2024-04-15T15:19:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.638+0000 7f25f3400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec0b4e00 con 0x7f25ec148110 2024-04-15T15:19:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.638+0000 7f25f3400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f25ec148670 con 0x7f25ec0af1f0 2024-04-15T15:19:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.638+0000 7f25ebe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0b43b0 0x7f25ec0cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.639+0000 7f25ebe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0b43b0 0x7f25ec0cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56492/0 (socket says 172.21.15.49:56492) 2024-04-15T15:19:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.639+0000 7f25ebe00640 1 -- 172.21.15.49:0/806094096 learned_addr learned my addr 172.21.15.49:0/806094096 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.639+0000 7f25f1400640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25ec148110 0x7f25ec13edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.639+0000 7f25f0a00640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25ec0af1f0 0x7f25ec146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.639+0000 7f25ebe00640 1 -- 172.21.15.49:0/806094096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25ec0af1f0 msgr2=0x7f25ec146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.639+0000 7f25ebe00640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25ec0af1f0 0x7f25ec146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.639+0000 7f25ebe00640 1 -- 172.21.15.49:0/806094096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25ec148110 msgr2=0x7f25ec13edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.639+0000 7f25ebe00640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25ec148110 0x7f25ec13edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.639+0000 7f25ebe00640 1 -- 172.21.15.49:0/806094096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec13dd30 con 0x7f25ec0b43b0 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.639+0000 7f25ebe00640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0b43b0 0x7f25ec0cf560 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f25d400d620 tx=0x7f25d400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.640+0000 7f25e8c00640 1 -- 172.21.15.49:0/806094096 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d400b620 con 0x7f25ec0b43b0 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.640+0000 7f25e8c00640 1 -- 172.21.15.49:0/806094096 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d4019070 con 0x7f25ec0b43b0 2024-04-15T15:19:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.640+0000 7f25e8c00640 1 -- 172.21.15.49:0/806094096 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d400ba00 con 0x7f25ec0b43b0 2024-04-15T15:19:53.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.640+0000 7f25f3400640 1 -- 172.21.15.49:0/806094096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec13e010 con 0x7f25ec0b43b0 2024-04-15T15:19:53.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.640+0000 7f25f3400640 1 -- 172.21.15.49:0/806094096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec13e5d0 con 0x7f25ec0b43b0 2024-04-15T15:19:53.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.641+0000 7f25f3400640 1 -- 172.21.15.49:0/806094096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ec068c80 con 0x7f25ec0b43b0 2024-04-15T15:19:53.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.642+0000 7f25e8c00640 1 -- 172.21.15.49:0/806094096 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25d400bba0 con 0x7f25ec0b43b0 2024-04-15T15:19:53.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.643+0000 7f25e8c00640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f25c806c8f0 0x7f25c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:53.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.643+0000 7f25e8c00640 1 -- 172.21.15.49:0/806094096 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f25d408bc00 con 0x7f25ec0b43b0 2024-04-15T15:19:53.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.643+0000 7f25f0a00640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f25c806c8f0 0x7f25c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:53.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.644+0000 7f25f0a00640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f25c806c8f0 0x7f25c806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000f60 tx=0x7f25dc007cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:53.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.648+0000 7f25e8c00640 1 -- 172.21.15.49:0/806094096 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25d405a370 con 0x7f25ec0b43b0 2024-04-15T15:19:53.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.924+0000 7f25f3400640 1 -- 172.21.15.49:0/806094096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f25ec0cce20 con 0x7f25ec0b43b0 2024-04-15T15:19:53.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.928+0000 7f25e8c00640 1 -- 172.21.15.49:0/806094096 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f25d4060070 con 0x7f25ec0b43b0 2024-04-15T15:19:53.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 -- 172.21.15.49:0/806094096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f25c806c8f0 msgr2=0x7f25c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f25c806c8f0 0x7f25c806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000f60 tx=0x7f25dc007cc0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 -- 172.21.15.49:0/806094096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0b43b0 msgr2=0x7f25ec0cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:53.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0b43b0 0x7f25ec0cf560 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f25d400d620 tx=0x7f25d400dae0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 -- 172.21.15.49:0/806094096 shutdown_connections 2024-04-15T15:19:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f25c806c8f0 0x7f25c806edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25ec148110 0x7f25ec13edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25ec0b43b0 0x7f25ec0cf560 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 --2- 172.21.15.49:0/806094096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25ec0af1f0 0x7f25ec146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 -- 172.21.15.49:0/806094096 >> 172.21.15.49:0/806094096 conn(0x7f25ec0b2460 msgr2=0x7f25ec074dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 -- 172.21.15.49:0/806094096 shutdown_connections 2024-04-15T15:19:53.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:53.931+0000 7f25f3400640 1 -- 172.21.15.49:0/806094096 wait complete. 2024-04-15T15:19:53.953 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-15T15:19:54.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.182+0000 7fdce1200640 1 Processor -- start 2024-04-15T15:19:54.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.182+0000 7fdce1200640 1 -- start start 2024-04-15T15:19:54.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.184+0000 7fdce1200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdcdc0b0cb0 0x7fdcdc0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.184+0000 7fdce1200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdcdc15d770 0x7fdcdc0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.184+0000 7fdce1200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcdc0cd940 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.184+0000 7fdce1200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fdcdc0ce550 con 0x7fdcdc0b0cb0 2024-04-15T15:19:54.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.184+0000 7fdce1200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdcdc0cdc20 con 0x7fdcdc15d770 2024-04-15T15:19:54.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcdb400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdcdc0b0cb0 0x7fdcdc0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcdbe00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdcdc0b59b0 0x7fdcdc15dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:52028/0 (socket says 172.21.15.49:52028) 2024-04-15T15:19:54.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcdbe00640 1 -- 172.21.15.49:0/1929409510 learned_addr learned my addr 172.21.15.49:0/1929409510 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:54.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcdaa00640 1 --2- 172.21.15.49:0/1929409510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdcdc15d770 0x7fdcdc0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcda000640 1 -- 172.21.15.49:0/1929409510 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2247861946 0 0) 0x7fdcdc0cd940 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcda000640 1 -- 172.21.15.49:0/1929409510 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcc0003520 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcdb400640 1 -- 172.21.15.49:0/1929409510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdcdc15d770 msgr2=0x7fdcdc0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:54.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcdb400640 1 --2- 172.21.15.49:0/1929409510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdcdc15d770 0x7fdcdc0cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcdb400640 1 -- 172.21.15.49:0/1929409510 >> v1:172.21.15.49:6789/0 conn(0x7fdcdc0b59b0 legacy=0x7fdcdc15dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcdb400640 1 -- 172.21.15.49:0/1929409510 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc0cdf00 con 0x7fdcdc0b0cb0 2024-04-15T15:19:54.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.185+0000 7fdcdaa00640 1 --2- 172.21.15.49:0/1929409510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdcdc15d770 0x7fdcdc0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:54.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.186+0000 7fdcdb400640 1 --2- 172.21.15.49:0/1929409510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b0cb0 0x7fdcdc0ce9e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0097a0 tx=0x7fdccc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45087a9f4b37b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:54.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.187+0000 7fdcda000640 1 -- 172.21.15.49:0/1929409510 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc035020 con 0x7fdcdc0b0cb0 2024-04-15T15:19:54.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.187+0000 7fdcda000640 1 -- 172.21.15.49:0/1929409510 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdccc031810 con 0x7fdcdc0b0cb0 2024-04-15T15:19:54.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.187+0000 7fdcda000640 1 -- 172.21.15.49:0/1929409510 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc031be0 con 0x7fdcdc0b0cb0 2024-04-15T15:19:54.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.187+0000 7fdce1200640 1 -- 172.21.15.49:0/1929409510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b0cb0 msgr2=0x7fdcdc0ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.187+0000 7fdce1200640 1 --2- 172.21.15.49:0/1929409510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b0cb0 0x7fdcdc0ce9e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0097a0 tx=0x7fdccc02b8b0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.188+0000 7fdce1200640 1 -- 172.21.15.49:0/1929409510 shutdown_connections 2024-04-15T15:19:54.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.188+0000 7fdce1200640 1 --2- 172.21.15.49:0/1929409510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdcdc15d770 0x7fdcdc0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.188+0000 7fdce1200640 1 --2- 172.21.15.49:0/1929409510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b0cb0 0x7fdcdc0ce9e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.188+0000 7fdce1200640 1 -- 172.21.15.49:0/1929409510 >> 172.21.15.49:0/1929409510 conn(0x7fdcdc0b23a0 msgr2=0x7fdcdc0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:54.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.188+0000 7fdce1200640 1 -- 172.21.15.49:0/1929409510 shutdown_connections 2024-04-15T15:19:54.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.188+0000 7fdce1200640 1 -- 172.21.15.49:0/1929409510 wait complete. 2024-04-15T15:19:54.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.189+0000 7fdce1200640 1 Processor -- start 2024-04-15T15:19:54.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.190+0000 7fdce1200640 1 -- start start 2024-04-15T15:19:54.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdce1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcdc0b0cb0 0x7fdcdc15e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdce1200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b59b0 0x7fdcdc15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdce1200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdcdc15d770 0x7fdcdc153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdce1200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc0cef20 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdce1200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc0b1090 con 0x7fdcdc0b0cb0 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdce1200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdcdc0b37a0 con 0x7fdcdc15d770 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdcdaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b59b0 0x7fdcdc15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdcdaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b59b0 0x7fdcdc15dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49886/0 (socket says 172.21.15.49:49886) 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdcdaa00640 1 -- 172.21.15.49:0/2395800096 learned_addr learned my addr 172.21.15.49:0/2395800096 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdcdb400640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcdc0b0cb0 0x7fdcdc15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdcdaa00640 1 -- 172.21.15.49:0/2395800096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdcdc15d770 msgr2=0x7fdcdc153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdcdaa00640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdcdc15d770 0x7fdcdc153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdcdaa00640 1 -- 172.21.15.49:0/2395800096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcdc0b0cb0 msgr2=0x7fdcdc15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdcdaa00640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcdc0b0cb0 0x7fdcdc15e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdcdaa00640 1 -- 172.21.15.49:0/2395800096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc0b3a80 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.191+0000 7fdcdb400640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcdc0b0cb0 0x7fdcdc15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.192+0000 7fdcdaa00640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b59b0 0x7fdcdc15dc10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800d7d0 tx=0x7fdcc800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.192+0000 7fdcd3400640 1 -- 172.21.15.49:0/2395800096 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8004030 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.192+0000 7fdcd3400640 1 -- 172.21.15.49:0/2395800096 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc80041d0 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.192+0000 7fdce1200640 1 -- 172.21.15.49:0/2395800096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc0b3d60 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.192+0000 7fdcd3400640 1 -- 172.21.15.49:0/2395800096 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc800b6c0 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.192+0000 7fdce1200640 1 -- 172.21.15.49:0/2395800096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc0b4320 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.193+0000 7fdce1200640 1 -- 172.21.15.49:0/2395800096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc0755c0 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.194+0000 7fdcd3400640 1 -- 172.21.15.49:0/2395800096 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcc800b860 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.196+0000 7fdcd3400640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdcbc06c8f0 0x7fdcbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.196+0000 7fdcd3400640 1 -- 172.21.15.49:0/2395800096 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fdcc808bb30 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.196+0000 7fdcdb400640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdcbc06c8f0 0x7fdcbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.197+0000 7fdcdb400640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdcbc06c8f0 0x7fdcbc06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdccc036000 tx=0x7fdccc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:54.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.200+0000 7fdcd3400640 1 -- 172.21.15.49:0/2395800096 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdcc805a210 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.490+0000 7fdce1200640 1 -- 172.21.15.49:0/2395800096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fdcdc159c90 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.495+0000 7fdcd3400640 1 -- 172.21.15.49:0/2395800096 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fdcc8059b10 con 0x7fdcdc0b59b0 2024-04-15T15:19:54.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 -- 172.21.15.49:0/2395800096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdcbc06c8f0 msgr2=0x7fdcbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdcbc06c8f0 0x7fdcbc06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdccc036000 tx=0x7fdccc008030 comp rx=0 tx=0).stop 2024-04-15T15:19:54.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 -- 172.21.15.49:0/2395800096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b59b0 msgr2=0x7fdcdc15dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b59b0 0x7fdcdc15dc10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800d7d0 tx=0x7fdcc800dc90 comp rx=0 tx=0).stop 2024-04-15T15:19:54.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 -- 172.21.15.49:0/2395800096 shutdown_connections 2024-04-15T15:19:54.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdcbc06c8f0 0x7fdcbc06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdcdc15d770 0x7fdcdc153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcdc0b59b0 0x7fdcdc15dc10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 --2- 172.21.15.49:0/2395800096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcdc0b0cb0 0x7fdcdc15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 -- 172.21.15.49:0/2395800096 >> 172.21.15.49:0/2395800096 conn(0x7fdcdc0b23a0 msgr2=0x7fdcdc15a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:54.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 -- 172.21.15.49:0/2395800096 shutdown_connections 2024-04-15T15:19:54.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.498+0000 7fdce1200640 1 -- 172.21.15.49:0/2395800096 wait complete. 2024-04-15T15:19:54.519 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-15T15:19:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.737+0000 7f520e800640 1 Processor -- start 2024-04-15T15:19:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.737+0000 7f520e800640 1 -- start start 2024-04-15T15:19:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.739+0000 7f520e800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f520815c9d0 0x7f52080d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.739+0000 7f520e800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f52080d0860 0x7f52080d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.739+0000 7f520e800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52080d1650 con 0x7f52080d1250 2024-04-15T15:19:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.739+0000 7f520e800640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5208070410 con 0x7f52080d0860 2024-04-15T15:19:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.739+0000 7f520e800640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f520809b8f0 con 0x7f520815c9d0 2024-04-15T15:19:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.739+0000 7f5206a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f52080d0860 0x7f52080d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.739+0000 7f5206a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f52080d0860 0x7f52080d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49898/0 (socket says 172.21.15.49:49898) 2024-04-15T15:19:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.739+0000 7f5206a00640 1 -- 172.21.15.49:0/3030577375 learned_addr learned my addr 172.21.15.49:0/3030577375 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.740+0000 7f5207400640 1 --2- 172.21.15.49:0/3030577375 >> v2:172.21.15.107:3300/0 conn(0x7f520815c9d0 0x7f52080d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.740+0000 7f5206a00640 1 -- 172.21.15.49:0/3030577375 >> v2:172.21.15.107:3300/0 conn(0x7f520815c9d0 msgr2=0x7f52080d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.740+0000 7f5206a00640 1 --2- 172.21.15.49:0/3030577375 >> v2:172.21.15.107:3300/0 conn(0x7f520815c9d0 0x7f52080d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.740+0000 7f5206a00640 1 -- 172.21.15.49:0/3030577375 >> v1:172.21.15.107:6789/0 conn(0x7f52080d1250 legacy=0x7f52080cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.740+0000 7f5206a00640 1 -- 172.21.15.49:0/3030577375 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520809bbd0 con 0x7f52080d0860 2024-04-15T15:19:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.740+0000 7f5207400640 1 --2- 172.21.15.49:0/3030577375 >> v2:172.21.15.107:3300/0 conn(0x7f520815c9d0 0x7f52080d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.741+0000 7f5206a00640 1 --2- 172.21.15.49:0/3030577375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d0860 0x7f52080d0d10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f52080737c0 tx=0x7f51f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636b4d42a58f0af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.741+0000 7f5206000640 1 -- 172.21.15.49:0/3030577375 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4035020 con 0x7f52080d0860 2024-04-15T15:19:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.741+0000 7f5206000640 1 -- 172.21.15.49:0/3030577375 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f4031870 con 0x7f52080d0860 2024-04-15T15:19:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.741+0000 7f5206000640 1 -- 172.21.15.49:0/3030577375 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4031c40 con 0x7f52080d0860 2024-04-15T15:19:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.742+0000 7f520e800640 1 -- 172.21.15.49:0/3030577375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d0860 msgr2=0x7f52080d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.742+0000 7f520e800640 1 --2- 172.21.15.49:0/3030577375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d0860 0x7f52080d0d10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f52080737c0 tx=0x7f51f402b910 comp rx=0 tx=0).stop 2024-04-15T15:19:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.742+0000 7f520e800640 1 -- 172.21.15.49:0/3030577375 shutdown_connections 2024-04-15T15:19:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.742+0000 7f520e800640 1 --2- 172.21.15.49:0/3030577375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d0860 0x7f52080d0d10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.742+0000 7f520e800640 1 --2- 172.21.15.49:0/3030577375 >> v2:172.21.15.107:3300/0 conn(0x7f520815c9d0 0x7f52080d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.742+0000 7f520e800640 1 -- 172.21.15.49:0/3030577375 >> 172.21.15.49:0/3030577375 conn(0x7f52080d2750 msgr2=0x7f52080d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.742+0000 7f520e800640 1 -- 172.21.15.49:0/3030577375 shutdown_connections 2024-04-15T15:19:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.742+0000 7f520e800640 1 -- 172.21.15.49:0/3030577375 wait complete. 2024-04-15T15:19:54.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.744+0000 7f520e800640 1 Processor -- start 2024-04-15T15:19:54.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.744+0000 7f520e800640 1 -- start start 2024-04-15T15:19:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.745+0000 7f520e800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52080d0860 0x7f5208153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.745+0000 7f520e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d1250 0x7f520806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.745+0000 7f520e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520815c9d0 0x7f520806cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.745+0000 7f520e800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f520809b5a0 con 0x7f52080d1250 2024-04-15T15:19:54.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.745+0000 7f520e800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5208070660 con 0x7f520815c9d0 2024-04-15T15:19:54.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.745+0000 7f520e800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f520815cdb0 con 0x7f52080d0860 2024-04-15T15:19:54.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5206a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d1250 0x7f520806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5206a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d1250 0x7f520806ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49900/0 (socket says 172.21.15.49:49900) 2024-04-15T15:19:54.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5206a00640 1 -- 172.21.15.49:0/1303602188 learned_addr learned my addr 172.21.15.49:0/1303602188 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:54.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5207400640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52080d0860 0x7f5208153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5207e00640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520815c9d0 0x7f520806cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5206a00640 1 -- 172.21.15.49:0/1303602188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52080d0860 msgr2=0x7f5208153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5206a00640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52080d0860 0x7f5208153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5206a00640 1 -- 172.21.15.49:0/1303602188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520815c9d0 msgr2=0x7f520806cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:54.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5206a00640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520815c9d0 0x7f520806cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:54.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5206a00640 1 -- 172.21.15.49:0/1303602188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52081f47f0 con 0x7f52080d1250 2024-04-15T15:19:54.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.746+0000 7f5207e00640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520815c9d0 0x7f520806cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:54.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.747+0000 7f5206a00640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d1250 0x7f520806ca40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f51f402b950 tx=0x7f51f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:54.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.747+0000 7f51ff400640 1 -- 172.21.15.49:0/1303602188 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4040070 con 0x7f52080d1250 2024-04-15T15:19:54.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.747+0000 7f51ff400640 1 -- 172.21.15.49:0/1303602188 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f403a420 con 0x7f52080d1250 2024-04-15T15:19:54.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.747+0000 7f51ff400640 1 -- 172.21.15.49:0/1303602188 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4035040 con 0x7f52080d1250 2024-04-15T15:19:54.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.747+0000 7f520e800640 1 -- 172.21.15.49:0/1303602188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520815fd70 con 0x7f52080d1250 2024-04-15T15:19:54.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.747+0000 7f520e800640 1 -- 172.21.15.49:0/1303602188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52080c6890 con 0x7f52080d1250 2024-04-15T15:19:54.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.748+0000 7f520e800640 1 -- 172.21.15.49:0/1303602188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5208068c80 con 0x7f52080d1250 2024-04-15T15:19:54.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.750+0000 7f51ff400640 1 -- 172.21.15.49:0/1303602188 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51f4038020 con 0x7f52080d1250 2024-04-15T15:19:54.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.750+0000 7f51ff400640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f51e806c920 0x7f51e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:54.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.751+0000 7f51ff400640 1 -- 172.21.15.49:0/1303602188 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f51f40ad890 con 0x7f52080d1250 2024-04-15T15:19:54.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.751+0000 7f5207400640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f51e806c920 0x7f51e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:54.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.751+0000 7f5207400640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f51e806c920 0x7f51e806ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f51f800a950 tx=0x7f51f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:54.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:54.755+0000 7f51ff400640 1 -- 172.21.15.49:0/1303602188 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51f407bf70 con 0x7f52080d1250 2024-04-15T15:19:55.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.039+0000 7f520e800640 1 -- 172.21.15.49:0/1303602188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f520815c0b0 con 0x7f52080d1250 2024-04-15T15:19:55.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.043+0000 7f51ff400640 1 -- 172.21.15.49:0/1303602188 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f51f407b870 con 0x7f52080d1250 2024-04-15T15:19:55.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.046+0000 7f520e800640 1 -- 172.21.15.49:0/1303602188 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f51e806c920 msgr2=0x7f51e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.046+0000 7f520e800640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f51e806c920 0x7f51e806ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f51f800a950 tx=0x7f51f8008040 comp rx=0 tx=0).stop 2024-04-15T15:19:55.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.046+0000 7f520e800640 1 -- 172.21.15.49:0/1303602188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d1250 msgr2=0x7f520806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.046+0000 7f520e800640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d1250 0x7f520806ca40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f51f402b950 tx=0x7f51f40046c0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.046+0000 7f520e800640 1 -- 172.21.15.49:0/1303602188 shutdown_connections 2024-04-15T15:19:55.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.046+0000 7f520e800640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f51e806c920 0x7f51e806ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.047+0000 7f520e800640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520815c9d0 0x7f520806cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.047+0000 7f520e800640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52080d1250 0x7f520806ca40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.047+0000 7f520e800640 1 --2- 172.21.15.49:0/1303602188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52080d0860 0x7f5208153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.047+0000 7f520e800640 1 -- 172.21.15.49:0/1303602188 >> 172.21.15.49:0/1303602188 conn(0x7f52080d2750 msgr2=0x7f52080b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:55.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.047+0000 7f520e800640 1 -- 172.21.15.49:0/1303602188 shutdown_connections 2024-04-15T15:19:55.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.047+0000 7f520e800640 1 -- 172.21.15.49:0/1303602188 wait complete. 2024-04-15T15:19:55.069 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-15T15:19:55.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.295+0000 7fe2a7200640 1 Processor -- start 2024-04-15T15:19:55.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.295+0000 7fe2a7200640 1 -- start start 2024-04-15T15:19:55.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.298+0000 7fe2a7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 0x7fe2a80b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.299+0000 7fe2a7200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a80cd940 con 0x7fe2a815d770 2024-04-15T15:19:55.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.299+0000 7fe2a7200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a80cdc20 con 0x7fe2a80b0cb0 2024-04-15T15:19:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.299+0000 7fe2a7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a80ce550 con 0x7fe2a80b59b0 2024-04-15T15:19:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.299+0000 7fe29fe00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe2a815d770 0x7fe2a80cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:52044/0 (socket says 172.21.15.49:52044) 2024-04-15T15:19:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.299+0000 7fe29fe00640 1 -- 172.21.15.49:0/1460763756 learned_addr learned my addr 172.21.15.49:0/1460763756 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.299+0000 7fe2a6800640 1 --2- 172.21.15.49:0/1460763756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 0x7fe2a80b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.299+0000 7fe2a6800640 1 -- 172.21.15.49:0/1460763756 >> v1:172.21.15.107:6789/0 conn(0x7fe2a80b0cb0 legacy=0x7fe2a80ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.299+0000 7fe2a6800640 1 -- 172.21.15.49:0/1460763756 >> v1:172.21.15.49:6789/0 conn(0x7fe2a815d770 legacy=0x7fe2a80cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.299+0000 7fe2a6800640 1 -- 172.21.15.49:0/1460763756 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a80cdf00 con 0x7fe2a80b59b0 2024-04-15T15:19:55.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.300+0000 7fe2a6800640 1 --2- 172.21.15.49:0/1460763756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 0x7fe2a80b5e60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009dc0 tx=0x7fe2a002bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddbb0945cdd2f42a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.306+0000 7fe2a5400640 1 -- 172.21.15.49:0/1460763756 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0039070 con 0x7fe2a80b59b0 2024-04-15T15:19:55.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.306+0000 7fe2a5400640 1 -- 172.21.15.49:0/1460763756 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a002ccc0 con 0x7fe2a80b59b0 2024-04-15T15:19:55.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.306+0000 7fe2a5400640 1 -- 172.21.15.49:0/1460763756 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0034660 con 0x7fe2a80b59b0 2024-04-15T15:19:55.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.306+0000 7fe2a7200640 1 -- 172.21.15.49:0/1460763756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 msgr2=0x7fe2a80b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.306+0000 7fe2a7200640 1 --2- 172.21.15.49:0/1460763756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 0x7fe2a80b5e60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009dc0 tx=0x7fe2a002bf10 comp rx=0 tx=0).stop 2024-04-15T15:19:55.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.306+0000 7fe2a7200640 1 -- 172.21.15.49:0/1460763756 shutdown_connections 2024-04-15T15:19:55.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.306+0000 7fe2a7200640 1 --2- 172.21.15.49:0/1460763756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 0x7fe2a80b5e60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.306+0000 7fe2a7200640 1 -- 172.21.15.49:0/1460763756 >> 172.21.15.49:0/1460763756 conn(0x7fe2a80b23a0 msgr2=0x7fe2a80b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:55.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.306+0000 7fe2a7200640 1 -- 172.21.15.49:0/1460763756 shutdown_connections 2024-04-15T15:19:55.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.306+0000 7fe2a7200640 1 -- 172.21.15.49:0/1460763756 wait complete. 2024-04-15T15:19:55.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a7200640 1 Processor -- start 2024-04-15T15:19:55.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a7200640 1 -- start start 2024-04-15T15:19:55.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a80b0cb0 0x7fe2a8160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 0x7fe2a8147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a7200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2a815d770 0x7fe2a815e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a7200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a815dc70 con 0x7fe2a815d770 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a80b1090 con 0x7fe2a80b0cb0 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a815ddf0 con 0x7fe2a80b59b0 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2a815d770 0x7fe2a815e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2a815d770 0x7fe2a815e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49906/0 (socket says 172.21.15.49:49906) 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.309+0000 7fe2a6800640 1 -- 172.21.15.49:0/3704796464 learned_addr learned my addr 172.21.15.49:0/3704796464 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.310+0000 7fe29fe00640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 0x7fe2a8147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.310+0000 7fe2a5e00640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a80b0cb0 0x7fe2a8160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.310+0000 7fe2a6800640 1 -- 172.21.15.49:0/3704796464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 msgr2=0x7fe2a8147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.310+0000 7fe2a6800640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 0x7fe2a8147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.310+0000 7fe2a6800640 1 -- 172.21.15.49:0/3704796464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a80b0cb0 msgr2=0x7fe2a8160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.310+0000 7fe2a6800640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a80b0cb0 0x7fe2a8160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.310+0000 7fe2a6800640 1 -- 172.21.15.49:0/3704796464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a81469f0 con 0x7fe2a815d770 2024-04-15T15:19:55.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.310+0000 7fe2a5e00640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a80b0cb0 0x7fe2a8160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:55.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.310+0000 7fe2a6800640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2a815d770 0x7fe2a815e440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009ee0 tx=0x7fe2a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.310+0000 7fe29e000640 1 -- 172.21.15.49:0/3704796464 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0044020 con 0x7fe2a815d770 2024-04-15T15:19:55.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.311+0000 7fe29e000640 1 -- 172.21.15.49:0/3704796464 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a003f470 con 0x7fe2a815d770 2024-04-15T15:19:55.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.311+0000 7fe29e000640 1 -- 172.21.15.49:0/3704796464 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0039040 con 0x7fe2a815d770 2024-04-15T15:19:55.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.311+0000 7fe2a7200640 1 -- 172.21.15.49:0/3704796464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a8146cd0 con 0x7fe2a815d770 2024-04-15T15:19:55.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.311+0000 7fe2a7200640 1 -- 172.21.15.49:0/3704796464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a8147290 con 0x7fe2a815d770 2024-04-15T15:19:55.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.313+0000 7fe2a7200640 1 -- 172.21.15.49:0/3704796464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a81682d0 con 0x7fe2a815d770 2024-04-15T15:19:55.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.313+0000 7fe29e000640 1 -- 172.21.15.49:0/3704796464 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2a0047020 con 0x7fe2a815d770 2024-04-15T15:19:55.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.314+0000 7fe29e000640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe28406c8f0 0x7fe28406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.314+0000 7fe29e000640 1 -- 172.21.15.49:0/3704796464 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fe2a00ae4a0 con 0x7fe2a815d770 2024-04-15T15:19:55.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.314+0000 7fe2a5e00640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe28406c8f0 0x7fe28406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.315+0000 7fe2a5e00640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe28406c8f0 0x7fe28406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80b5560 tx=0x7fe298009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.321+0000 7fe29e000640 1 -- 172.21.15.49:0/3704796464 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2a004e030 con 0x7fe2a815d770 2024-04-15T15:19:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.607+0000 7fe2a7200640 1 -- 172.21.15.49:0/3704796464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fe2a80b3040 con 0x7fe2a815d770 2024-04-15T15:19:55.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.612+0000 7fe29e000640 1 -- 172.21.15.49:0/3704796464 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fe2a007cb80 con 0x7fe2a815d770 2024-04-15T15:19:55.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.615+0000 7fe2a7200640 1 -- 172.21.15.49:0/3704796464 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe28406c8f0 msgr2=0x7fe28406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.615+0000 7fe2a7200640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe28406c8f0 0x7fe28406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80b5560 tx=0x7fe298009040 comp rx=0 tx=0).stop 2024-04-15T15:19:55.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.615+0000 7fe2a7200640 1 -- 172.21.15.49:0/3704796464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2a815d770 msgr2=0x7fe2a815e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.615+0000 7fe2a7200640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2a815d770 0x7fe2a815e440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009ee0 tx=0x7fe2a0004a40 comp rx=0 tx=0).stop 2024-04-15T15:19:55.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.616+0000 7fe2a7200640 1 -- 172.21.15.49:0/3704796464 shutdown_connections 2024-04-15T15:19:55.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.616+0000 7fe2a7200640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe28406c8f0 0x7fe28406edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.616+0000 7fe2a7200640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2a80b59b0 0x7fe2a8147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.616+0000 7fe2a7200640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2a815d770 0x7fe2a815e440 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.616+0000 7fe2a7200640 1 --2- 172.21.15.49:0/3704796464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a80b0cb0 0x7fe2a8160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.616+0000 7fe2a7200640 1 -- 172.21.15.49:0/3704796464 >> 172.21.15.49:0/3704796464 conn(0x7fe2a80b23a0 msgr2=0x7fe2a8148c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:55.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.616+0000 7fe2a7200640 1 -- 172.21.15.49:0/3704796464 shutdown_connections 2024-04-15T15:19:55.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.616+0000 7fe2a7200640 1 -- 172.21.15.49:0/3704796464 wait complete. 2024-04-15T15:19:55.636 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-15T15:19:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.854+0000 7fdbc8200640 1 Processor -- start 2024-04-15T15:19:55.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.855+0000 7fdbc8200640 1 -- start start 2024-04-15T15:19:55.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.856+0000 7fdbc8200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 0x7fdbc00ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.856+0000 7fdbc8200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdbc0072580 0x7fdbc006e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.856+0000 7fdbc8200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbc015d200 con 0x7fdbc0072100 2024-04-15T15:19:55.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.856+0000 7fdbc8200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc00a95b0 con 0x7fdbc0072580 2024-04-15T15:19:55.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.856+0000 7fdbc8200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc015d4e0 con 0x7fdbc013f6a0 2024-04-15T15:19:55.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.857+0000 7fdbc6200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdbc0072580 0x7fdbc006e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.857+0000 7fdbc5800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 0x7fdbc00ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.857+0000 7fdbc5800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 0x7fdbc00ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:59070/0 (socket says 172.21.15.49:59070) 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.857+0000 7fdbc5800640 1 -- 172.21.15.49:0/189820913 learned_addr learned my addr 172.21.15.49:0/189820913 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.857+0000 7fdbc6200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdbc0072580 0x7fdbc006e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:33664/0 (socket says 172.21.15.49:33664) 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.858+0000 7fdbc5800640 1 -- 172.21.15.49:0/189820913 >> v2:172.21.15.107:3300/0 conn(0x7fdbc0072580 msgr2=0x7fdbc006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.858+0000 7fdbc5800640 1 --2- 172.21.15.49:0/189820913 >> v2:172.21.15.107:3300/0 conn(0x7fdbc0072580 0x7fdbc006e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.858+0000 7fdbc5800640 1 -- 172.21.15.49:0/189820913 >> v1:172.21.15.107:6789/0 conn(0x7fdbc0072100 legacy=0x7fdbc006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.858+0000 7fdbc5800640 1 -- 172.21.15.49:0/189820913 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc015d7c0 con 0x7fdbc013f6a0 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.859+0000 7fdbc5800640 1 --2- 172.21.15.49:0/189820913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 0x7fdbc00ceda0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009870 tx=0x7fdbb402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83dfa4300fac58b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.860+0000 7fdbbfe00640 1 -- 172.21.15.49:0/189820913 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4034070 con 0x7fdbc013f6a0 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.860+0000 7fdbbfe00640 1 -- 172.21.15.49:0/189820913 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb4030840 con 0x7fdbc013f6a0 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.860+0000 7fdbbfe00640 1 -- 172.21.15.49:0/189820913 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4030c10 con 0x7fdbc013f6a0 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.860+0000 7fdbc8200640 1 -- 172.21.15.49:0/189820913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 msgr2=0x7fdbc00ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.860+0000 7fdbc8200640 1 --2- 172.21.15.49:0/189820913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 0x7fdbc00ceda0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009870 tx=0x7fdbb402b980 comp rx=0 tx=0).stop 2024-04-15T15:19:55.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.860+0000 7fdbc8200640 1 -- 172.21.15.49:0/189820913 shutdown_connections 2024-04-15T15:19:55.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.860+0000 7fdbc8200640 1 --2- 172.21.15.49:0/189820913 >> v2:172.21.15.107:3300/0 conn(0x7fdbc0072580 0x7fdbc006e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.860+0000 7fdbc8200640 1 --2- 172.21.15.49:0/189820913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 0x7fdbc00ceda0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.860+0000 7fdbc8200640 1 -- 172.21.15.49:0/189820913 >> 172.21.15.49:0/189820913 conn(0x7fdbc00b2460 msgr2=0x7fdbc00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:55.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.861+0000 7fdbc8200640 1 -- 172.21.15.49:0/189820913 shutdown_connections 2024-04-15T15:19:55.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.861+0000 7fdbc8200640 1 -- 172.21.15.49:0/189820913 wait complete. 2024-04-15T15:19:55.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.862+0000 7fdbc8200640 1 Processor -- start 2024-04-15T15:19:55.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.862+0000 7fdbc8200640 1 -- start start 2024-04-15T15:19:55.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.863+0000 7fdbc8200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc0072100 0x7fdbc0158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.863+0000 7fdbc8200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbc0072580 0x7fdbc00aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.863+0000 7fdbc8200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 0x7fdbc00ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.863+0000 7fdbc8200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc013fef0 con 0x7fdbc0072100 2024-04-15T15:19:55.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.863+0000 7fdbc8200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc00729f0 con 0x7fdbc0072580 2024-04-15T15:19:55.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.863+0000 7fdbc8200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc015dd80 con 0x7fdbc013f6a0 2024-04-15T15:19:55.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc5800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc0072100 0x7fdbc0158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc5800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc0072100 0x7fdbc0158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49912/0 (socket says 172.21.15.49:49912) 2024-04-15T15:19:55.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc5800640 1 -- 172.21.15.49:0/700664803 learned_addr learned my addr 172.21.15.49:0/700664803 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:55.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc6200640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 0x7fdbc00ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc4e00640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbc0072580 0x7fdbc00aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc5800640 1 -- 172.21.15.49:0/700664803 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 msgr2=0x7fdbc00ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc5800640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 0x7fdbc00ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc5800640 1 -- 172.21.15.49:0/700664803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbc0072580 msgr2=0x7fdbc00aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:55.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc5800640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbc0072580 0x7fdbc00aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:55.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc5800640 1 -- 172.21.15.49:0/700664803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc00aba90 con 0x7fdbc0072100 2024-04-15T15:19:55.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.864+0000 7fdbc4e00640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbc0072580 0x7fdbc00aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:19:55.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.865+0000 7fdbc5800640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc0072100 0x7fdbc0158bc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4000c00 tx=0x7fdbb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.865+0000 7fdbbd600640 1 -- 172.21.15.49:0/700664803 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4048070 con 0x7fdbc0072100 2024-04-15T15:19:55.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.865+0000 7fdbbd600640 1 -- 172.21.15.49:0/700664803 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb4030640 con 0x7fdbc0072100 2024-04-15T15:19:55.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.865+0000 7fdbc8200640 1 -- 172.21.15.49:0/700664803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc00ac050 con 0x7fdbc0072100 2024-04-15T15:19:55.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.865+0000 7fdbbd600640 1 -- 172.21.15.49:0/700664803 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4034040 con 0x7fdbc0072100 2024-04-15T15:19:55.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.866+0000 7fdbc8200640 1 -- 172.21.15.49:0/700664803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc00ac610 con 0x7fdbc0072100 2024-04-15T15:19:55.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.867+0000 7fdbc8200640 1 -- 172.21.15.49:0/700664803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc00ab800 con 0x7fdbc0072100 2024-04-15T15:19:55.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.868+0000 7fdbbd600640 1 -- 172.21.15.49:0/700664803 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbb404b070 con 0x7fdbc0072100 2024-04-15T15:19:55.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.869+0000 7fdbbd600640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdba006c8f0 0x7fdba006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:55.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.869+0000 7fdbbd600640 1 -- 172.21.15.49:0/700664803 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fdbb40ad6b0 con 0x7fdbc0072100 2024-04-15T15:19:55.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.870+0000 7fdbc4e00640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdba006c8f0 0x7fdba006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:55.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.870+0000 7fdbc4e00640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdba006c8f0 0x7fdba006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00b22c0 tx=0x7fdbac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:55.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:55.874+0000 7fdbbd600640 1 -- 172.21.15.49:0/700664803 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbb407b740 con 0x7fdbc0072100 2024-04-15T15:19:56.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.154+0000 7fdbc8200640 1 -- 172.21.15.49:0/700664803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fdbc0068c80 con 0x7fdbc0072100 2024-04-15T15:19:56.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.158+0000 7fdbbd600640 1 -- 172.21.15.49:0/700664803 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7fdbb407fa40 con 0x7fdbc0072100 2024-04-15T15:19:56.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.161+0000 7fdbc8200640 1 -- 172.21.15.49:0/700664803 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdba006c8f0 msgr2=0x7fdba006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.161+0000 7fdbc8200640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdba006c8f0 0x7fdba006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00b22c0 tx=0x7fdbac009040 comp rx=0 tx=0).stop 2024-04-15T15:19:56.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.161+0000 7fdbc8200640 1 -- 172.21.15.49:0/700664803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc0072100 msgr2=0x7fdbc0158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.161+0000 7fdbc8200640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc0072100 0x7fdbc0158bc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4000c00 tx=0x7fdbb4004730 comp rx=0 tx=0).stop 2024-04-15T15:19:56.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.161+0000 7fdbc8200640 1 -- 172.21.15.49:0/700664803 shutdown_connections 2024-04-15T15:19:56.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.161+0000 7fdbc8200640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdba006c8f0 0x7fdba006edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.161+0000 7fdbc8200640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbc013f6a0 0x7fdbc00ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.161+0000 7fdbc8200640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbc0072580 0x7fdbc00aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.161+0000 7fdbc8200640 1 --2- 172.21.15.49:0/700664803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc0072100 0x7fdbc0158bc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.161+0000 7fdbc8200640 1 -- 172.21.15.49:0/700664803 >> 172.21.15.49:0/700664803 conn(0x7fdbc00b2460 msgr2=0x7fdbc01415f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:56.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.162+0000 7fdbc8200640 1 -- 172.21.15.49:0/700664803 shutdown_connections 2024-04-15T15:19:56.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.162+0000 7fdbc8200640 1 -- 172.21.15.49:0/700664803 wait complete. 2024-04-15T15:19:56.183 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:19:56.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.396+0000 7f77e8c00640 1 Processor -- start 2024-04-15T15:19:56.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.396+0000 7f77e8c00640 1 -- start start 2024-04-15T15:19:56.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.398+0000 7f77e8c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f77e415b860 0x7f77e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.398+0000 7f77e8c00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e40cdfb0 con 0x7f77e415b480 2024-04-15T15:19:56.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.398+0000 7f77e8c00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e415f2b0 con 0x7f77e406f320 2024-04-15T15:19:56.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.398+0000 7f77e8c00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f77e415f060 con 0x7f77e415b860 2024-04-15T15:19:56.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.398+0000 7f77e3400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f77e415b480 0x7f77e40cd8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:52056/0 (socket says 172.21.15.49:52056) 2024-04-15T15:19:56.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.398+0000 7f77e3400640 1 -- 172.21.15.49:0/1450395657 learned_addr learned my addr 172.21.15.49:0/1450395657 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:56.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.398+0000 7f77e2a00640 1 --2- 172.21.15.49:0/1450395657 >> v2:172.21.15.107:3300/0 conn(0x7f77e415b860 0x7f77e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.398+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 447306751 0 0) 0x7f77e40cdfb0 con 0x7f77e415b480 2024-04-15T15:19:56.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77c8003520 con 0x7f77e415b480 2024-04-15T15:19:56.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 70687913 0 0) 0x7f77c8003520 con 0x7f77e415b480 2024-04-15T15:19:56.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 >> v2:172.21.15.107:3300/0 conn(0x7f77e415b860 msgr2=0x7f77e406ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 --2- 172.21.15.49:0/1450395657 >> v2:172.21.15.107:3300/0 conn(0x7f77e415b860 0x7f77e406ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 >> v1:172.21.15.107:6789/0 conn(0x7f77e406f320 legacy=0x7f77e40b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e415f590 con 0x7f77e415b480 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f77d40019f0 con 0x7f77e415b480 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 >> v1:172.21.15.49:6789/0 conn(0x7f77e415b480 legacy=0x7f77e40cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c80039b0 0x7f77c8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77c8004800 0x7f77c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c80051c0 0x7f77c800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77e415f060 con 0x7f77c80039b0 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77c800e250 con 0x7f77c80051c0 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f77c800e450 con 0x7f77c8004800 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.399+0000 7f77e3400640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c80039b0 0x7f77c8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.400+0000 7f77e2a00640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77c8004800 0x7f77c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.400+0000 7f77e3400640 1 -- 172.21.15.49:0/1450395657 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77c8004800 msgr2=0x7f77c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.400+0000 7f77e3400640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77c8004800 0x7f77c8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.400+0000 7f77e3400640 1 -- 172.21.15.49:0/1450395657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c80051c0 msgr2=0x7f77c800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.400+0000 7f77e3e00640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c80051c0 0x7f77c800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.400+0000 7f77e3400640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c80051c0 0x7f77c800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.400+0000 7f77e3400640 1 -- 172.21.15.49:0/1450395657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c8003520 con 0x7f77c80039b0 2024-04-15T15:19:56.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.400+0000 7f77e3e00640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77c80051c0 0x7f77c800dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:56.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.400+0000 7f77e3e00640 1 -- 172.21.15.49:0/1450395657 reap_dead start 2024-04-15T15:19:56.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.401+0000 7f77e3400640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c80039b0 0x7f77c8003db0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f77d400a860 tx=0x7f77c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef517c99e080f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.401+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d400a480 con 0x7f77c80039b0 2024-04-15T15:19:56.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.401+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d4034630 con 0x7f77c80039b0 2024-04-15T15:19:56.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.401+0000 7f77e2000640 1 -- 172.21.15.49:0/1450395657 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4034aa0 con 0x7f77c80039b0 2024-04-15T15:19:56.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.402+0000 7f77e8c00640 1 -- 172.21.15.49:0/1450395657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c80039b0 msgr2=0x7f77c8003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.402+0000 7f77e8c00640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c80039b0 0x7f77c8003db0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f77d400a860 tx=0x7f77c8005830 comp rx=0 tx=0).stop 2024-04-15T15:19:56.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.402+0000 7f77e8c00640 1 -- 172.21.15.49:0/1450395657 shutdown_connections 2024-04-15T15:19:56.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.402+0000 7f77e8c00640 1 --2- 172.21.15.49:0/1450395657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c80039b0 0x7f77c8003db0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.402+0000 7f77e8c00640 1 -- 172.21.15.49:0/1450395657 >> 172.21.15.49:0/1450395657 conn(0x7f77e406b930 msgr2=0x7f77e406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:56.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.402+0000 7f77e8c00640 1 -- 172.21.15.49:0/1450395657 shutdown_connections 2024-04-15T15:19:56.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.402+0000 7f77e8c00640 1 -- 172.21.15.49:0/1450395657 wait complete. 2024-04-15T15:19:56.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.404+0000 7f77e8c00640 1 Processor -- start 2024-04-15T15:19:56.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.404+0000 7f77e8c00640 1 -- start start 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e8c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77c80039b0 0x7f77e40cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e8c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e414a890 0x7f77e40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e8c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e4155fe0 0x7f77e4156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e8c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77e415fb50 con 0x7f77e4155fe0 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e8c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77e415fec0 con 0x7f77e414a890 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e8c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f77e40cd550 con 0x7f77c80039b0 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e4155fe0 0x7f77e4156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e4155fe0 0x7f77e4156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49924/0 (socket says 172.21.15.49:49924) 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e3e00640 1 -- 172.21.15.49:0/920249676 learned_addr learned my addr 172.21.15.49:0/920249676 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e3400640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77c80039b0 0x7f77e40cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e2a00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e414a890 0x7f77e40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e3e00640 1 -- 172.21.15.49:0/920249676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77c80039b0 msgr2=0x7f77e40cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e3e00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77c80039b0 0x7f77e40cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e3e00640 1 -- 172.21.15.49:0/920249676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e414a890 msgr2=0x7f77e40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e3e00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e414a890 0x7f77e40cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e3e00640 1 -- 172.21.15.49:0/920249676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e40cbf20 con 0x7f77e4155fe0 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.405+0000 7f77e2a00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e414a890 0x7f77e40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.406+0000 7f77e3e00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e4155fe0 0x7f77e4156460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f77e406ff90 tx=0x7f77dc00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.406+0000 7f77db400640 1 -- 172.21.15.49:0/920249676 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc005050 con 0x7f77e4155fe0 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.406+0000 7f77db400640 1 -- 172.21.15.49:0/920249676 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77dc005210 con 0x7f77e4155fe0 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.406+0000 7f77db400640 1 -- 172.21.15.49:0/920249676 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc007020 con 0x7f77e4155fe0 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.406+0000 7f77e8c00640 1 -- 172.21.15.49:0/920249676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e415c6d0 con 0x7f77e4155fe0 2024-04-15T15:19:56.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.406+0000 7f77e8c00640 1 -- 172.21.15.49:0/920249676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e415a420 con 0x7f77e4155fe0 2024-04-15T15:19:56.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.407+0000 7f77e8c00640 1 -- 172.21.15.49:0/920249676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e41569a0 con 0x7f77e4155fe0 2024-04-15T15:19:56.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.408+0000 7f77db400640 1 -- 172.21.15.49:0/920249676 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77dc019020 con 0x7f77e4155fe0 2024-04-15T15:19:56.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.409+0000 7f77db400640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f77c406c8f0 0x7f77c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.409+0000 7f77db400640 1 -- 172.21.15.49:0/920249676 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f77dc08abb0 con 0x7f77e4155fe0 2024-04-15T15:19:56.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.409+0000 7f77e3400640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f77c406c8f0 0x7f77c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.410+0000 7f77e3400640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f77c406c8f0 0x7f77c406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f77d4034000 tx=0x7f77d4036040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.413+0000 7f77db400640 1 -- 172.21.15.49:0/920249676 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77e40cbf20 con 0x7f77e4155fe0 2024-04-15T15:19:56.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.617+0000 7f77e8c00640 1 -- 172.21.15.49:0/920249676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77e415cbb0 con 0x7f77e4155fe0 2024-04-15T15:19:56.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.619+0000 7f77db400640 1 -- 172.21.15.49:0/920249676 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+12891 (secure 0 0 0) 0x7f77e415cbb0 con 0x7f77e4155fe0 2024-04-15T15:19:56.621 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:56.621 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":31,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:19:52.303833+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.621+0000 7f77e8c00640 1 -- 172.21.15.49:0/920249676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f77c406c8f0 msgr2=0x7f77c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.621+0000 7f77e8c00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f77c406c8f0 0x7f77c406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f77d4034000 tx=0x7f77d4036040 comp rx=0 tx=0).stop 2024-04-15T15:19:56.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.621+0000 7f77e8c00640 1 -- 172.21.15.49:0/920249676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e4155fe0 msgr2=0x7f77e4156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.621+0000 7f77e8c00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e4155fe0 0x7f77e4156460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f77e406ff90 tx=0x7f77dc00d890 comp rx=0 tx=0).stop 2024-04-15T15:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.622+0000 7f77e8c00640 1 -- 172.21.15.49:0/920249676 shutdown_connections 2024-04-15T15:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.622+0000 7f77e8c00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f77c406c8f0 0x7f77c406edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.622+0000 7f77e8c00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e4155fe0 0x7f77e4156460 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.622+0000 7f77e8c00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e414a890 0x7f77e40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.622+0000 7f77e8c00640 1 --2- 172.21.15.49:0/920249676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77c80039b0 0x7f77e40cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.622+0000 7f77e8c00640 1 -- 172.21.15.49:0/920249676 >> 172.21.15.49:0/920249676 conn(0x7f77e406b930 msgr2=0x7f77e406eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.622+0000 7f77e8c00640 1 -- 172.21.15.49:0/920249676 shutdown_connections 2024-04-15T15:19:56.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.622+0000 7f77e8c00640 1 -- 172.21.15.49:0/920249676 wait complete. 2024-04-15T15:19:56.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:19:56.644 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:19:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.875+0000 7fd67ce00640 1 Processor -- start 2024-04-15T15:19:56.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.875+0000 7fd67ce00640 1 -- start start 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.876+0000 7fd67ce00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd6780d1fc0 0x7fd6780ba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.876+0000 7fd67ce00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6780d3140 con 0x7fd67815ed60 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.876+0000 7fd67ce00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6780d3420 con 0x7fd6780d2b80 2024-04-15T15:19:56.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.876+0000 7fd67ce00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd6780ba7b0 con 0x7fd6780d1fc0 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.877+0000 7fd676800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd6780d1fc0 0x7fd6780ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.877+0000 7fd675e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd67815ed60 0x7fd6780d41e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:52064/0 (socket says 172.21.15.49:52064) 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.877+0000 7fd675e00640 1 -- 172.21.15.49:0/1645278584 learned_addr learned my addr 172.21.15.49:0/1645278584 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.877+0000 7fd676800640 1 -- 172.21.15.49:0/1645278584 >> v1:172.21.15.107:6789/0 conn(0x7fd6780d2b80 legacy=0x7fd6780d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.877+0000 7fd676800640 1 -- 172.21.15.49:0/1645278584 >> v1:172.21.15.49:6789/0 conn(0x7fd67815ed60 legacy=0x7fd6780d41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.877+0000 7fd676800640 1 -- 172.21.15.49:0/1645278584 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6780d3700 con 0x7fd6780d1fc0 2024-04-15T15:19:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.878+0000 7fd676800640 1 --2- 172.21.15.49:0/1645278584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d1fc0 0x7fd6780ba270 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd670009d90 tx=0x7fd67002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4a7e9a8c557d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.879+0000 7fd675400640 1 -- 172.21.15.49:0/1645278584 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670039070 con 0x7fd6780d1fc0 2024-04-15T15:19:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.879+0000 7fd675400640 1 -- 172.21.15.49:0/1645278584 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67002cc60 con 0x7fd6780d1fc0 2024-04-15T15:19:56.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.879+0000 7fd675400640 1 -- 172.21.15.49:0/1645278584 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670034660 con 0x7fd6780d1fc0 2024-04-15T15:19:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.879+0000 7fd67ce00640 1 -- 172.21.15.49:0/1645278584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d1fc0 msgr2=0x7fd6780ba270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.879+0000 7fd67ce00640 1 --2- 172.21.15.49:0/1645278584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d1fc0 0x7fd6780ba270 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd670009d90 tx=0x7fd67002beb0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.879+0000 7fd67ce00640 1 -- 172.21.15.49:0/1645278584 shutdown_connections 2024-04-15T15:19:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.879+0000 7fd67ce00640 1 --2- 172.21.15.49:0/1645278584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d1fc0 0x7fd6780ba270 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.879+0000 7fd67ce00640 1 -- 172.21.15.49:0/1645278584 >> 172.21.15.49:0/1645278584 conn(0x7fd67806d3e0 msgr2=0x7fd67806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.880+0000 7fd67ce00640 1 -- 172.21.15.49:0/1645278584 shutdown_connections 2024-04-15T15:19:56.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.880+0000 7fd67ce00640 1 -- 172.21.15.49:0/1645278584 wait complete. 2024-04-15T15:19:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.881+0000 7fd67ce00640 1 Processor -- start 2024-04-15T15:19:56.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.881+0000 7fd67ce00640 1 -- start start 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.882+0000 7fd67ce00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6780d1fc0 0x7fd6781559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.882+0000 7fd67ce00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d2b80 0x7fd6780b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.882+0000 7fd67ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd67815ed60 0x7fd6780b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.882+0000 7fd67ce00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd67815f140 con 0x7fd6780d2b80 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.882+0000 7fd67ce00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780d3cc0 con 0x7fd67815ed60 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.882+0000 7fd67ce00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd6780b7a80 con 0x7fd6780d1fc0 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.883+0000 7fd66d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d2b80 0x7fd6780b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.883+0000 7fd66d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d2b80 0x7fd6780b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49938/0 (socket says 172.21.15.49:49938) 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.883+0000 7fd66d400640 1 -- 172.21.15.49:0/65017445 learned_addr learned my addr 172.21.15.49:0/65017445 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.883+0000 7fd676800640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd67815ed60 0x7fd6780b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.883+0000 7fd675e00640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6780d1fc0 0x7fd6781559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.883+0000 7fd66d400640 1 -- 172.21.15.49:0/65017445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6780d1fc0 msgr2=0x7fd6781559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.883+0000 7fd66d400640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6780d1fc0 0x7fd6781559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.883+0000 7fd66d400640 1 -- 172.21.15.49:0/65017445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd67815ed60 msgr2=0x7fd6780b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.883+0000 7fd66d400640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd67815ed60 0x7fd6780b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.883+0000 7fd66d400640 1 -- 172.21.15.49:0/65017445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd678158340 con 0x7fd6780d2b80 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.884+0000 7fd66d400640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d2b80 0x7fd6780b5430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c460 tx=0x7fd66000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.884+0000 7fd66ea00640 1 -- 172.21.15.49:0/65017445 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660016020 con 0x7fd6780d2b80 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.884+0000 7fd66ea00640 1 -- 172.21.15.49:0/65017445 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd660012870 con 0x7fd6780d2b80 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.884+0000 7fd66ea00640 1 -- 172.21.15.49:0/65017445 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660012c60 con 0x7fd6780d2b80 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.884+0000 7fd67ce00640 1 -- 172.21.15.49:0/65017445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd678158620 con 0x7fd6780d2b80 2024-04-15T15:19:56.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.884+0000 7fd67ce00640 1 -- 172.21.15.49:0/65017445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd678158be0 con 0x7fd6780d2b80 2024-04-15T15:19:56.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.885+0000 7fd67ce00640 1 -- 172.21.15.49:0/65017445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd678068c80 con 0x7fd6780d2b80 2024-04-15T15:19:56.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.886+0000 7fd66ea00640 1 -- 172.21.15.49:0/65017445 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd660019020 con 0x7fd6780d2b80 2024-04-15T15:19:56.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.887+0000 7fd66ea00640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd65406c8f0 0x7fd65406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:56.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.888+0000 7fd66ea00640 1 -- 172.21.15.49:0/65017445 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fd66008b290 con 0x7fd6780d2b80 2024-04-15T15:19:56.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.888+0000 7fd675e00640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd65406c8f0 0x7fd65406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:56.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.888+0000 7fd675e00640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd65406c8f0 0x7fd65406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd6780cdaf0 tx=0x7fd668009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:56.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:56.892+0000 7fd66ea00640 1 -- 172.21.15.49:0/65017445 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd660020070 con 0x7fd6780d2b80 2024-04-15T15:19:57.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.096+0000 7fd67ce00640 1 -- 172.21.15.49:0/65017445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd678162040 con 0x7fd6780d2b80 2024-04-15T15:19:57.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.097+0000 7fd66ea00640 1 -- 172.21.15.49:0/65017445 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) ==== 93+0+10 (secure 0 0 0) 0x7fd660059a00 con 0x7fd6780d2b80 2024-04-15T15:19:57.102 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:19:57.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.100+0000 7fd67ce00640 1 -- 172.21.15.49:0/65017445 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd65406c8f0 msgr2=0x7fd65406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.100+0000 7fd67ce00640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd65406c8f0 0x7fd65406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd6780cdaf0 tx=0x7fd668009040 comp rx=0 tx=0).stop 2024-04-15T15:19:57.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.100+0000 7fd67ce00640 1 -- 172.21.15.49:0/65017445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d2b80 msgr2=0x7fd6780b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.100+0000 7fd67ce00640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d2b80 0x7fd6780b5430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c460 tx=0x7fd66000c920 comp rx=0 tx=0).stop 2024-04-15T15:19:57.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.100+0000 7fd67ce00640 1 -- 172.21.15.49:0/65017445 shutdown_connections 2024-04-15T15:19:57.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.100+0000 7fd67ce00640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd65406c8f0 0x7fd65406edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.100+0000 7fd67ce00640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd67815ed60 0x7fd6780b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.100+0000 7fd67ce00640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6780d2b80 0x7fd6780b5430 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.100+0000 7fd67ce00640 1 --2- 172.21.15.49:0/65017445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6780d1fc0 0x7fd6781559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.100+0000 7fd67ce00640 1 -- 172.21.15.49:0/65017445 >> 172.21.15.49:0/65017445 conn(0x7fd67806d3e0 msgr2=0x7fd6780b9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.101+0000 7fd67ce00640 1 -- 172.21.15.49:0/65017445 shutdown_connections 2024-04-15T15:19:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.101+0000 7fd67ce00640 1 -- 172.21.15.49:0/65017445 wait complete. 2024-04-15T15:19:57.122 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-15T15:19:57.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.349+0000 7fefd0a00640 1 Processor -- start 2024-04-15T15:19:57.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.349+0000 7fefd0a00640 1 -- start start 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.350+0000 7fefd0a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fefcc0cee00 0x7fefcc06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.350+0000 7fefd0a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefcc071f50 0x7fefcc0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.350+0000 7fefd0a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fefcc15cbe0 0x7fefcc15d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.350+0000 7fefd0a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fefcc06f4f0 con 0x7fefcc0cee00 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.350+0000 7fefd0a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fefcc0728f0 con 0x7fefcc15cbe0 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.350+0000 7fefd0a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc15dbb0 con 0x7fefcc071f50 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.351+0000 7fefca800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fefcc0cee00 0x7fefcc06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.351+0000 7fefca800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fefcc0cee00 0x7fefcc06e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49952/0 (socket says 172.21.15.49:49952) 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.351+0000 7fefca800640 1 -- 172.21.15.49:0/677945873 learned_addr learned my addr 172.21.15.49:0/677945873 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.351+0000 7fefcb200640 1 --2- 172.21.15.49:0/677945873 >> v2:172.21.15.107:3300/0 conn(0x7fefcc15cbe0 0x7fefcc15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.351+0000 7fefc9e00640 1 --2- 172.21.15.49:0/677945873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefcc071f50 0x7fefcc0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.351+0000 7fefca800640 1 -- 172.21.15.49:0/677945873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefcc071f50 msgr2=0x7fefcc0723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:57.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.352+0000 7fefca800640 1 --2- 172.21.15.49:0/677945873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefcc071f50 0x7fefcc0723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.352+0000 7fefca800640 1 -- 172.21.15.49:0/677945873 >> v2:172.21.15.107:3300/0 conn(0x7fefcc15cbe0 msgr2=0x7fefcc15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:57.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.352+0000 7fefca800640 1 --2- 172.21.15.49:0/677945873 >> v2:172.21.15.107:3300/0 conn(0x7fefcc15cbe0 0x7fefcc15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.352+0000 7fefca800640 1 -- 172.21.15.49:0/677945873 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc15d8d0 con 0x7fefcc0cee00 2024-04-15T15:19:57.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.353+0000 7fefca800640 1 --2- 172.21.15.49:0/677945873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 0x7fefcc06e8c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009870 tx=0x7fefbc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cd18de17037338 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.353+0000 7fefc9400640 1 -- 172.21.15.49:0/677945873 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc035020 con 0x7fefcc0cee00 2024-04-15T15:19:57.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.353+0000 7fefc9400640 1 -- 172.21.15.49:0/677945873 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc031640 con 0x7fefcc0cee00 2024-04-15T15:19:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.353+0000 7fefc9400640 1 -- 172.21.15.49:0/677945873 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc02d040 con 0x7fefcc0cee00 2024-04-15T15:19:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.354+0000 7fefd0a00640 1 -- 172.21.15.49:0/677945873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 msgr2=0x7fefcc06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.354+0000 7fefd0a00640 1 --2- 172.21.15.49:0/677945873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 0x7fefcc06e8c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009870 tx=0x7fefbc02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.354+0000 7fefd0a00640 1 -- 172.21.15.49:0/677945873 shutdown_connections 2024-04-15T15:19:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.354+0000 7fefd0a00640 1 --2- 172.21.15.49:0/677945873 >> v2:172.21.15.107:3300/0 conn(0x7fefcc15cbe0 0x7fefcc15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.354+0000 7fefd0a00640 1 --2- 172.21.15.49:0/677945873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefcc071f50 0x7fefcc0723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.354+0000 7fefd0a00640 1 --2- 172.21.15.49:0/677945873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 0x7fefcc06e8c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.354+0000 7fefd0a00640 1 -- 172.21.15.49:0/677945873 >> 172.21.15.49:0/677945873 conn(0x7fefcc06b930 msgr2=0x7fefcc06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.354+0000 7fefd0a00640 1 -- 172.21.15.49:0/677945873 shutdown_connections 2024-04-15T15:19:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.354+0000 7fefd0a00640 1 -- 172.21.15.49:0/677945873 wait complete. 2024-04-15T15:19:57.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.356+0000 7fefd0a00640 1 Processor -- start 2024-04-15T15:19:57.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.356+0000 7fefd0a00640 1 -- start start 2024-04-15T15:19:57.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.357+0000 7fefd0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefcc071f50 0x7fefcc14eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.357+0000 7fefd0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 0x7fefcc1f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.357+0000 7fefd0a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefcc15cbe0 0x7fefcc1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.357+0000 7fefd0a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc06e3a0 con 0x7fefcc0cee00 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.357+0000 7fefd0a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc06e0c0 con 0x7fefcc071f50 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.357+0000 7fefd0a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc15d5b0 con 0x7fefcc15cbe0 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.357+0000 7fefc9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 0x7fefcc1f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.357+0000 7fefc9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 0x7fefcc1f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49954/0 (socket says 172.21.15.49:49954) 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.357+0000 7fefc9e00640 1 -- 172.21.15.49:0/2878538810 learned_addr learned my addr 172.21.15.49:0/2878538810 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.358+0000 7fefca800640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefcc071f50 0x7fefcc14eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.358+0000 7fefcb200640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefcc15cbe0 0x7fefcc1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.358+0000 7fefc9e00640 1 -- 172.21.15.49:0/2878538810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefcc15cbe0 msgr2=0x7fefcc1500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.358+0000 7fefc9e00640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefcc15cbe0 0x7fefcc1500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.358+0000 7fefc9e00640 1 -- 172.21.15.49:0/2878538810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefcc071f50 msgr2=0x7fefcc14eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.358+0000 7fefc9e00640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefcc071f50 0x7fefcc14eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.358+0000 7fefc9e00640 1 -- 172.21.15.49:0/2878538810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc15ef50 con 0x7fefcc0cee00 2024-04-15T15:19:57.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.358+0000 7fefc9e00640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 0x7fefcc1f4900 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fefb800d5f0 tx=0x7fefb800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.359+0000 7fefc2a00640 1 -- 172.21.15.49:0/2878538810 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb80125f0 con 0x7fefcc0cee00 2024-04-15T15:19:57.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.359+0000 7fefc2a00640 1 -- 172.21.15.49:0/2878538810 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb8019070 con 0x7fefcc0cee00 2024-04-15T15:19:57.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.359+0000 7fefc2a00640 1 -- 172.21.15.49:0/2878538810 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb80129d0 con 0x7fefcc0cee00 2024-04-15T15:19:57.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.359+0000 7fefd0a00640 1 -- 172.21.15.49:0/2878538810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefcc15f230 con 0x7fefcc0cee00 2024-04-15T15:19:57.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.359+0000 7fefd0a00640 1 -- 172.21.15.49:0/2878538810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefcc14f060 con 0x7fefcc0cee00 2024-04-15T15:19:57.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.360+0000 7fefd0a00640 1 -- 172.21.15.49:0/2878538810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefcc0755c0 con 0x7fefcc0cee00 2024-04-15T15:19:57.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.361+0000 7fefc2a00640 1 -- 172.21.15.49:0/2878538810 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefb8012b70 con 0x7fefcc0cee00 2024-04-15T15:19:57.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.362+0000 7fefc2a00640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fefac06c8f0 0x7fefac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.362+0000 7fefc2a00640 1 -- 172.21.15.49:0/2878538810 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fefb808bc60 con 0x7fefcc0cee00 2024-04-15T15:19:57.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.362+0000 7fefca800640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fefac06c8f0 0x7fefac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.363+0000 7fefca800640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fefac06c8f0 0x7fefac06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fefbc036000 tx=0x7fefbc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.366+0000 7fefc2a00640 1 -- 172.21.15.49:0/2878538810 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefb805a3d0 con 0x7fefcc0cee00 2024-04-15T15:19:57.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.572+0000 7fefd0a00640 1 -- 172.21.15.49:0/2878538810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fefcc0b5140 con 0x7fefcc0cee00 2024-04-15T15:19:57.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.573+0000 7fefc2a00640 1 -- 172.21.15.49:0/2878538810 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) ==== 84+0+65 (secure 0 0 0) 0x7fefb8059cd0 con 0x7fefcc0cee00 2024-04-15T15:19:57.574 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:19:57.575 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2024-04-15T15:19:57.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:19:57.575 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2024-04-15T15:19:57.575 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2024-04-15T15:19:57.575 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:19:57.575 INFO:teuthology.orchestra.run.smithi049.stdout:] 2024-04-15T15:19:57.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.575+0000 7fefd0a00640 1 -- 172.21.15.49:0/2878538810 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fefac06c8f0 msgr2=0x7fefac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.575+0000 7fefd0a00640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fefac06c8f0 0x7fefac06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fefbc036000 tx=0x7fefbc03c040 comp rx=0 tx=0).stop 2024-04-15T15:19:57.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.575+0000 7fefd0a00640 1 -- 172.21.15.49:0/2878538810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 msgr2=0x7fefcc1f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.575+0000 7fefd0a00640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 0x7fefcc1f4900 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fefb800d5f0 tx=0x7fefb800dab0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.575+0000 7fefd0a00640 1 -- 172.21.15.49:0/2878538810 shutdown_connections 2024-04-15T15:19:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.575+0000 7fefd0a00640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fefac06c8f0 0x7fefac06edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.575+0000 7fefd0a00640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefcc15cbe0 0x7fefcc1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.575+0000 7fefd0a00640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefcc0cee00 0x7fefcc1f4900 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.575+0000 7fefd0a00640 1 --2- 172.21.15.49:0/2878538810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefcc071f50 0x7fefcc14eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.575+0000 7fefd0a00640 1 -- 172.21.15.49:0/2878538810 >> 172.21.15.49:0/2878538810 conn(0x7fefcc06b930 msgr2=0x7fefcc15a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.576+0000 7fefd0a00640 1 -- 172.21.15.49:0/2878538810 shutdown_connections 2024-04-15T15:19:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.576+0000 7fefd0a00640 1 -- 172.21.15.49:0/2878538810 wait complete. 2024-04-15T15:19:57.598 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-15T15:19:57.598 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-15T15:19:57.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.826+0000 7f729ca00640 1 Processor -- start 2024-04-15T15:19:57.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.827+0000 7f729ca00640 1 -- start start 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.828+0000 7f729ca00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f729815ecf0 0x7f72980d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.828+0000 7f729ca00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f72980d30b0 0x7f72980d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.828+0000 7f729ca00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72980d4eb0 0x7f72980b3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.828+0000 7f729ca00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f729815f0d0 con 0x7f72980d4eb0 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.828+0000 7f729ca00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f72980b7f40 con 0x7f72980d30b0 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.828+0000 7f729ca00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f72980b7930 con 0x7f729815ecf0 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.829+0000 7f7296800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72980d4eb0 0x7f72980b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.829+0000 7f7296800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72980d4eb0 0x7f72980b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49964/0 (socket says 172.21.15.49:49964) 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.829+0000 7f7296800640 1 -- 172.21.15.49:0/3943768056 learned_addr learned my addr 172.21.15.49:0/3943768056 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.829+0000 7f7295e00640 1 --2- 172.21.15.49:0/3943768056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f729815ecf0 0x7f72980d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.829+0000 7f728fe00640 1 --2- 172.21.15.49:0/3943768056 >> v2:172.21.15.107:3300/0 conn(0x7f72980d30b0 0x7f72980d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.829+0000 7f7296800640 1 -- 172.21.15.49:0/3943768056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f729815ecf0 msgr2=0x7f72980d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:57.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.829+0000 7f7296800640 1 --2- 172.21.15.49:0/3943768056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f729815ecf0 0x7f72980d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.829+0000 7f7296800640 1 -- 172.21.15.49:0/3943768056 >> v2:172.21.15.107:3300/0 conn(0x7f72980d30b0 msgr2=0x7f72980d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:57.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.829+0000 7f7296800640 1 --2- 172.21.15.49:0/3943768056 >> v2:172.21.15.107:3300/0 conn(0x7f72980d30b0 0x7f72980d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.829+0000 7f7296800640 1 -- 172.21.15.49:0/3943768056 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72980d3550 con 0x7f72980d4eb0 2024-04-15T15:19:57.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.830+0000 7f7296800640 1 --2- 172.21.15.49:0/3943768056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 0x7f72980b3550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f72980d2560 tx=0x7f729002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abaf7c206d599cd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.831+0000 7f7295400640 1 -- 172.21.15.49:0/3943768056 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729002ca70 con 0x7f72980d4eb0 2024-04-15T15:19:57.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.831+0000 7f7295400640 1 -- 172.21.15.49:0/3943768056 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f729002cc10 con 0x7f72980d4eb0 2024-04-15T15:19:57.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.831+0000 7f7295400640 1 -- 172.21.15.49:0/3943768056 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72900343f0 con 0x7f72980d4eb0 2024-04-15T15:19:57.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.831+0000 7f729ca00640 1 -- 172.21.15.49:0/3943768056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 msgr2=0x7f72980b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.831+0000 7f729ca00640 1 --2- 172.21.15.49:0/3943768056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 0x7f72980b3550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f72980d2560 tx=0x7f729002bed0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.832+0000 7f729ca00640 1 -- 172.21.15.49:0/3943768056 shutdown_connections 2024-04-15T15:19:57.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.832+0000 7f729ca00640 1 --2- 172.21.15.49:0/3943768056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 0x7f72980b3550 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.832+0000 7f729ca00640 1 --2- 172.21.15.49:0/3943768056 >> v2:172.21.15.107:3300/0 conn(0x7f72980d30b0 0x7f72980d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.832+0000 7f729ca00640 1 --2- 172.21.15.49:0/3943768056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f729815ecf0 0x7f72980d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.832+0000 7f729ca00640 1 -- 172.21.15.49:0/3943768056 >> 172.21.15.49:0/3943768056 conn(0x7f7298073480 msgr2=0x7f7298073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:57.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.832+0000 7f729ca00640 1 -- 172.21.15.49:0/3943768056 shutdown_connections 2024-04-15T15:19:57.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.832+0000 7f729ca00640 1 -- 172.21.15.49:0/3943768056 wait complete. 2024-04-15T15:19:57.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.833+0000 7f729ca00640 1 Processor -- start 2024-04-15T15:19:57.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.834+0000 7f729ca00640 1 -- start start 2024-04-15T15:19:57.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f729ca00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72980d30b0 0x7f72981f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f729ca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 0x7f7298152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f729ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f729815ecf0 0x7f72980ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f729ca00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f72980d3f80 con 0x7f72980d4eb0 2024-04-15T15:19:57.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f729ca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72980b7bd0 con 0x7f729815ecf0 2024-04-15T15:19:57.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f729ca00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f72980d4100 con 0x7f72980d30b0 2024-04-15T15:19:57.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f728fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 0x7f7298152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f728fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 0x7f7298152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49974/0 (socket says 172.21.15.49:49974) 2024-04-15T15:19:57.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f728fe00640 1 -- 172.21.15.49:0/2898123422 learned_addr learned my addr 172.21.15.49:0/2898123422 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:57.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f7295e00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72980d30b0 0x7f72981f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f7296800640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f729815ecf0 0x7f72980ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f728fe00640 1 -- 172.21.15.49:0/2898123422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72980d30b0 msgr2=0x7f72981f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f728fe00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72980d30b0 0x7f72981f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f728fe00640 1 -- 172.21.15.49:0/2898123422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f729815ecf0 msgr2=0x7f72980ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:57.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f728fe00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f729815ecf0 0x7f72980ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:57.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.835+0000 7f728fe00640 1 -- 172.21.15.49:0/2898123422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72980cbfb0 con 0x7f72980d4eb0 2024-04-15T15:19:57.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.836+0000 7f728fe00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 0x7f7298152de0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f728000d830 tx=0x7f728000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.836+0000 7f728e000640 1 -- 172.21.15.49:0/2898123422 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280018030 con 0x7f72980d4eb0 2024-04-15T15:19:57.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.836+0000 7f728e000640 1 -- 172.21.15.49:0/2898123422 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f728000be10 con 0x7f72980d4eb0 2024-04-15T15:19:57.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.836+0000 7f728e000640 1 -- 172.21.15.49:0/2898123422 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280009850 con 0x7f72980d4eb0 2024-04-15T15:19:57.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.836+0000 7f729ca00640 1 -- 172.21.15.49:0/2898123422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72980cc290 con 0x7f72980d4eb0 2024-04-15T15:19:57.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.836+0000 7f729ca00640 1 -- 172.21.15.49:0/2898123422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72980cc850 con 0x7f72980d4eb0 2024-04-15T15:19:57.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.838+0000 7f729ca00640 1 -- 172.21.15.49:0/2898123422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7298068db0 con 0x7f72980d4eb0 2024-04-15T15:19:57.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.839+0000 7f728e000640 1 -- 172.21.15.49:0/2898123422 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f728001b070 con 0x7f72980d4eb0 2024-04-15T15:19:57.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.839+0000 7f728e000640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f727406c8f0 0x7f727406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:57.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.840+0000 7f728e000640 1 -- 172.21.15.49:0/2898123422 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f728008ba00 con 0x7f72980d4eb0 2024-04-15T15:19:57.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.840+0000 7f7295e00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f727406c8f0 0x7f727406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:57.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.841+0000 7f7295e00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f727406c8f0 0x7f727406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:57.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:57.844+0000 7f728e000640 1 -- 172.21.15.49:0/2898123422 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f728005a0e0 con 0x7f72980d4eb0 2024-04-15T15:19:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.049+0000 7f729ca00640 1 -- 172.21.15.49:0/2898123422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f72981f41b0 con 0x7f72980d4eb0 2024-04-15T15:19:58.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.519+0000 7f728e000640 1 -- 172.21.15.49:0/2898123422 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) ==== 138+0+0 (secure 0 0 0) 0x7f72800599e0 con 0x7f72980d4eb0 2024-04-15T15:19:58.521 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2024-04-15T15:19:58.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.523+0000 7f729ca00640 1 -- 172.21.15.49:0/2898123422 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f727406c8f0 msgr2=0x7f727406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.523+0000 7f729ca00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f727406c8f0 0x7f727406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288008040 comp rx=0 tx=0).stop 2024-04-15T15:19:58.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.523+0000 7f729ca00640 1 -- 172.21.15.49:0/2898123422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 msgr2=0x7f7298152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.523+0000 7f729ca00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 0x7f7298152de0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f728000d830 tx=0x7f728000dcf0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.523+0000 7f729ca00640 1 -- 172.21.15.49:0/2898123422 shutdown_connections 2024-04-15T15:19:58.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.523+0000 7f729ca00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f727406c8f0 0x7f727406edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.523+0000 7f729ca00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f729815ecf0 0x7f72980ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.523+0000 7f729ca00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72980d4eb0 0x7f7298152de0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.523+0000 7f729ca00640 1 --2- 172.21.15.49:0/2898123422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72980d30b0 0x7f72981f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.523+0000 7f729ca00640 1 -- 172.21.15.49:0/2898123422 >> 172.21.15.49:0/2898123422 conn(0x7f7298073480 msgr2=0x7f72980b82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:58.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.524+0000 7f729ca00640 1 -- 172.21.15.49:0/2898123422 shutdown_connections 2024-04-15T15:19:58.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.524+0000 7f729ca00640 1 -- 172.21.15.49:0/2898123422 wait complete. 2024-04-15T15:19:58.553 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-15T15:19:58.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.761+0000 7f04c3200640 1 Processor -- start 2024-04-15T15:19:58.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.761+0000 7f04c3200640 1 -- start start 2024-04-15T15:19:58.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.763+0000 7f04c3200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f04bc15a440 0x7f04bc15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.763+0000 7f04c3200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04bc072150 0x7f04bc0725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.763+0000 7f04c3200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04bc15d520 con 0x7f04bc06fc80 2024-04-15T15:19:58.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.763+0000 7f04c3200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f04bc06e280 con 0x7f04bc15a440 2024-04-15T15:19:58.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.763+0000 7f04c3200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc15d800 con 0x7f04bc072150 2024-04-15T15:19:58.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.763+0000 7f04bbe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f04bc15a440 0x7f04bc15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.763+0000 7f04bb400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f04bc06fc80 0x7f04bc0cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:52074/0 (socket says 172.21.15.49:52074) 2024-04-15T15:19:58.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.763+0000 7f04bb400640 1 -- 172.21.15.49:0/1497777667 learned_addr learned my addr 172.21.15.49:0/1497777667 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:19:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.764+0000 7f04c1200640 1 --2- 172.21.15.49:0/1497777667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04bc072150 0x7f04bc0725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.764+0000 7f04bbe00640 1 -- 172.21.15.49:0/1497777667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04bc072150 msgr2=0x7f04bc0725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.764+0000 7f04bbe00640 1 --2- 172.21.15.49:0/1497777667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04bc072150 0x7f04bc0725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.764+0000 7f04bbe00640 1 -- 172.21.15.49:0/1497777667 >> v1:172.21.15.49:6789/0 conn(0x7f04bc06fc80 legacy=0x7f04bc0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.764+0000 7f04bbe00640 1 -- 172.21.15.49:0/1497777667 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc15dae0 con 0x7f04bc15a440 2024-04-15T15:19:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.764+0000 7f04baa00640 1 -- 172.21.15.49:0/1497777667 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1953557855 0 0) 0x7f04bc15d520 con 0x7f04bc06fc80 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.765+0000 7f04bbe00640 1 --2- 172.21.15.49:0/1497777667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 0x7f04bc15cef0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0097a0 tx=0x7f04ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c432d6ac3a8f6b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.765+0000 7f04baa00640 1 -- 172.21.15.49:0/1497777667 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac035020 con 0x7f04bc15a440 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.765+0000 7f04baa00640 1 -- 172.21.15.49:0/1497777667 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ac031830 con 0x7f04bc15a440 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.765+0000 7f04baa00640 1 -- 172.21.15.49:0/1497777667 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac031c00 con 0x7f04bc15a440 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.766+0000 7f04c3200640 1 -- 172.21.15.49:0/1497777667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 msgr2=0x7f04bc15cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.766+0000 7f04c3200640 1 --2- 172.21.15.49:0/1497777667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 0x7f04bc15cef0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0097a0 tx=0x7f04ac02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.766+0000 7f04c3200640 1 -- 172.21.15.49:0/1497777667 shutdown_connections 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.766+0000 7f04c3200640 1 --2- 172.21.15.49:0/1497777667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04bc072150 0x7f04bc0725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.766+0000 7f04c3200640 1 --2- 172.21.15.49:0/1497777667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 0x7f04bc15cef0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.766+0000 7f04c3200640 1 -- 172.21.15.49:0/1497777667 >> 172.21.15.49:0/1497777667 conn(0x7f04bc0b2460 msgr2=0x7f04bc0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.766+0000 7f04c3200640 1 -- 172.21.15.49:0/1497777667 shutdown_connections 2024-04-15T15:19:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.766+0000 7f04c3200640 1 -- 172.21.15.49:0/1497777667 wait complete. 2024-04-15T15:19:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.768+0000 7f04c3200640 1 Processor -- start 2024-04-15T15:19:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.768+0000 7f04c3200640 1 -- start start 2024-04-15T15:19:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.769+0000 7f04c3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04bc06fc80 0x7f04bc159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.769+0000 7f04c3200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04bc072150 0x7f04bc1e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.769+0000 7f04c3200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 0x7f04bc1e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.769+0000 7f04c3200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc15ddc0 con 0x7f04bc15a440 2024-04-15T15:19:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.769+0000 7f04c3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc15a820 con 0x7f04bc06fc80 2024-04-15T15:19:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.769+0000 7f04c3200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc06f510 con 0x7f04bc072150 2024-04-15T15:19:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.770+0000 7f04c1200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 0x7f04bc1e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.770+0000 7f04c1200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 0x7f04bc1e7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49988/0 (socket says 172.21.15.49:49988) 2024-04-15T15:19:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.770+0000 7f04c1200640 1 -- 172.21.15.49:0/2445660782 learned_addr learned my addr 172.21.15.49:0/2445660782 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.770+0000 7f04bb400640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04bc072150 0x7f04bc1e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.770+0000 7f04bbe00640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04bc06fc80 0x7f04bc159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.770+0000 7f04c1200640 1 -- 172.21.15.49:0/2445660782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04bc072150 msgr2=0x7f04bc1e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.770+0000 7f04c1200640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04bc072150 0x7f04bc1e77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.770+0000 7f04c1200640 1 -- 172.21.15.49:0/2445660782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04bc06fc80 msgr2=0x7f04bc159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.770+0000 7f04c1200640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04bc06fc80 0x7f04bc159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.770+0000 7f04c1200640 1 -- 172.21.15.49:0/2445660782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc1e7020 con 0x7f04bc15a440 2024-04-15T15:19:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.771+0000 7f04c1200640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 0x7f04bc1e7d20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f04b4007e70 tx=0x7f04b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.771+0000 7f04b3e00640 1 -- 172.21.15.49:0/2445660782 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4014620 con 0x7f04bc15a440 2024-04-15T15:19:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.771+0000 7f04b3e00640 1 -- 172.21.15.49:0/2445660782 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b4019070 con 0x7f04bc15a440 2024-04-15T15:19:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.771+0000 7f04b3e00640 1 -- 172.21.15.49:0/2445660782 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4014a20 con 0x7f04bc15a440 2024-04-15T15:19:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.771+0000 7f04c3200640 1 -- 172.21.15.49:0/2445660782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc1e5f80 con 0x7f04bc15a440 2024-04-15T15:19:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.771+0000 7f04c3200640 1 -- 172.21.15.49:0/2445660782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc1e6540 con 0x7f04bc15a440 2024-04-15T15:19:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.772+0000 7f04c3200640 1 -- 172.21.15.49:0/2445660782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc068c80 con 0x7f04bc15a440 2024-04-15T15:19:58.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.774+0000 7f04b3e00640 1 -- 172.21.15.49:0/2445660782 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04b4014bc0 con 0x7f04bc15a440 2024-04-15T15:19:58.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.775+0000 7f04b3e00640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f049806c8f0 0x7f049806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:58.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.775+0000 7f04b3e00640 1 -- 172.21.15.49:0/2445660782 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5227+0+0 (secure 0 0 0) 0x7f04b408c2d0 con 0x7f04bc15a440 2024-04-15T15:19:58.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.775+0000 7f04bbe00640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f049806c8f0 0x7f049806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:58.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.776+0000 7f04bbe00640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f049806c8f0 0x7f049806edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f04ac036000 tx=0x7f04ac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:58.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.780+0000 7f04b3e00640 1 -- 172.21.15.49:0/2445660782 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04b4011260 con 0x7f04bc15a440 2024-04-15T15:19:58.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:58.990+0000 7f04c3200640 1 -- 172.21.15.49:0/2445660782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f04bc159960 con 0x7f04bc15a440 2024-04-15T15:19:59.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.525+0000 7f04b3e00640 1 -- 172.21.15.49:0/2445660782 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) ==== 170+0+0 (secure 0 0 0) 0x7f04b405a8d0 con 0x7f04bc15a440 2024-04-15T15:19:59.527 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2024-04-15T15:19:59.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.529+0000 7f04c3200640 1 -- 172.21.15.49:0/2445660782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f049806c8f0 msgr2=0x7f049806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.529+0000 7f04c3200640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f049806c8f0 0x7f049806edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f04ac036000 tx=0x7f04ac008030 comp rx=0 tx=0).stop 2024-04-15T15:19:59.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.529+0000 7f04c3200640 1 -- 172.21.15.49:0/2445660782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 msgr2=0x7f04bc1e7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.529+0000 7f04c3200640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 0x7f04bc1e7d20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f04b4007e70 tx=0x7f04b400e4b0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.529+0000 7f04c3200640 1 -- 172.21.15.49:0/2445660782 shutdown_connections 2024-04-15T15:19:59.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.529+0000 7f04c3200640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f049806c8f0 0x7f049806edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.529+0000 7f04c3200640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04bc072150 0x7f04bc1e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.529+0000 7f04c3200640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04bc15a440 0x7f04bc1e7d20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.530+0000 7f04c3200640 1 --2- 172.21.15.49:0/2445660782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04bc06fc80 0x7f04bc159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.530+0000 7f04c3200640 1 -- 172.21.15.49:0/2445660782 >> 172.21.15.49:0/2445660782 conn(0x7f04bc0b2460 msgr2=0x7f04bc0b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:59.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.530+0000 7f04c3200640 1 -- 172.21.15.49:0/2445660782 shutdown_connections 2024-04-15T15:19:59.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.531+0000 7f04c3200640 1 -- 172.21.15.49:0/2445660782 wait complete. 2024-04-15T15:19:59.576 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-15T15:19:59.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.770+0000 7f7690a00640 1 Processor -- start 2024-04-15T15:19:59.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.771+0000 7f7690a00640 1 -- start start 2024-04-15T15:19:59.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.772+0000 7f7690a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f768c15e9a0 0x7f768c162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.772+0000 7f7690a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f768c0cfc00 0x7f768c0d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.772+0000 7f7690a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f768c163450 con 0x7f768c0ba3e0 2024-04-15T15:19:59.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.772+0000 7f7690a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f768c162e90 con 0x7f768c15e9a0 2024-04-15T15:19:59.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.772+0000 7f7690a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f768c163170 con 0x7f768c0cfc00 2024-04-15T15:19:59.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.773+0000 7f768a800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f768c15e9a0 0x7f768c162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.773+0000 7f768a800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f768c15e9a0 0x7f768c162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:33774/0 (socket says 172.21.15.49:33774) 2024-04-15T15:19:59.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.773+0000 7f768a800640 1 -- 172.21.15.49:0/2071398201 learned_addr learned my addr 172.21.15.49:0/2071398201 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:59.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.773+0000 7f768b200640 1 --2- 172.21.15.49:0/2071398201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f768c0cfc00 0x7f768c0d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.773+0000 7f768a800640 1 -- 172.21.15.49:0/2071398201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f768c0cfc00 msgr2=0x7f768c0d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:19:59.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.774+0000 7f768a800640 1 --2- 172.21.15.49:0/2071398201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f768c0cfc00 0x7f768c0d3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.774+0000 7f768a800640 1 -- 172.21.15.49:0/2071398201 >> v1:172.21.15.107:6789/0 conn(0x7f768c0ba3e0 legacy=0x7f768c0cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.774+0000 7f768a800640 1 -- 172.21.15.49:0/2071398201 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c0d2920 con 0x7f768c15e9a0 2024-04-15T15:19:59.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.775+0000 7f768a800640 1 --2- 172.21.15.49:0/2071398201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768c15e9a0 0x7f768c162590 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f767c009e50 tx=0x7f767c02bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bd8b1014f390847 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.775+0000 7f7689400640 1 -- 172.21.15.49:0/2071398201 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c02cb90 con 0x7f768c15e9a0 2024-04-15T15:19:59.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.776+0000 7f7689400640 1 -- 172.21.15.49:0/2071398201 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767c02cd30 con 0x7f768c15e9a0 2024-04-15T15:19:59.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.776+0000 7f7689400640 1 -- 172.21.15.49:0/2071398201 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c034610 con 0x7f768c15e9a0 2024-04-15T15:19:59.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.776+0000 7f7690a00640 1 -- 172.21.15.49:0/2071398201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768c15e9a0 msgr2=0x7f768c162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.776+0000 7f7690a00640 1 --2- 172.21.15.49:0/2071398201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768c15e9a0 0x7f768c162590 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f767c009e50 tx=0x7f767c02bff0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.776+0000 7f7690a00640 1 -- 172.21.15.49:0/2071398201 shutdown_connections 2024-04-15T15:19:59.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.776+0000 7f7690a00640 1 --2- 172.21.15.49:0/2071398201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f768c0cfc00 0x7f768c0d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.776+0000 7f7690a00640 1 --2- 172.21.15.49:0/2071398201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768c15e9a0 0x7f768c162590 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.776+0000 7f7690a00640 1 -- 172.21.15.49:0/2071398201 >> 172.21.15.49:0/2071398201 conn(0x7f768c06ce40 msgr2=0x7f768c06d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:19:59.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.776+0000 7f7690a00640 1 -- 172.21.15.49:0/2071398201 shutdown_connections 2024-04-15T15:19:59.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.776+0000 7f7690a00640 1 -- 172.21.15.49:0/2071398201 wait complete. 2024-04-15T15:19:59.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.778+0000 7f7690a00640 1 Processor -- start 2024-04-15T15:19:59.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.778+0000 7f7690a00640 1 -- start start 2024-04-15T15:19:59.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.779+0000 7f7690a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f768c0ba3e0 0x7f768c1618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.779+0000 7f7690a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768c0cfc00 0x7f768c140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.779+0000 7f7690a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f768c15e9a0 0x7f768c13fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.779+0000 7f7690a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f768c15ed80 con 0x7f768c15e9a0 2024-04-15T15:19:59.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.779+0000 7f7690a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f768c162ad0 con 0x7f768c0cfc00 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.779+0000 7f7690a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f768c13f800 con 0x7f768c0ba3e0 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.780+0000 7f768b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f768c15e9a0 0x7f768c13fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.780+0000 7f768b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f768c15e9a0 0x7f768c13fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50002/0 (socket says 172.21.15.49:50002) 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.780+0000 7f768b200640 1 -- 172.21.15.49:0/1717202766 learned_addr learned my addr 172.21.15.49:0/1717202766 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.780+0000 7f768a800640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f768c0ba3e0 0x7f768c1618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.780+0000 7f7689e00640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768c0cfc00 0x7f768c140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.780+0000 7f768b200640 1 -- 172.21.15.49:0/1717202766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f768c0ba3e0 msgr2=0x7f768c1618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.780+0000 7f768b200640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f768c0ba3e0 0x7f768c1618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.780+0000 7f768b200640 1 -- 172.21.15.49:0/1717202766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768c0cfc00 msgr2=0x7f768c140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.780+0000 7f768b200640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768c0cfc00 0x7f768c140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.780+0000 7f768b200640 1 -- 172.21.15.49:0/1717202766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c13ebf0 con 0x7f768c15e9a0 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.781+0000 7f768b200640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f768c15e9a0 0x7f768c13fc90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f768400e880 tx=0x7f768400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.781+0000 7f7682a00640 1 -- 172.21.15.49:0/1717202766 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684017070 con 0x7f768c15e9a0 2024-04-15T15:19:59.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.781+0000 7f7682a00640 1 -- 172.21.15.49:0/1717202766 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768400c940 con 0x7f768c15e9a0 2024-04-15T15:19:59.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.781+0000 7f7682a00640 1 -- 172.21.15.49:0/1717202766 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768400cd30 con 0x7f768c15e9a0 2024-04-15T15:19:59.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.781+0000 7f7690a00640 1 -- 172.21.15.49:0/1717202766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c13eed0 con 0x7f768c15e9a0 2024-04-15T15:19:59.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.781+0000 7f7690a00640 1 -- 172.21.15.49:0/1717202766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c13f490 con 0x7f768c15e9a0 2024-04-15T15:19:59.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.782+0000 7f7690a00640 1 -- 172.21.15.49:0/1717202766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768c0755c0 con 0x7f768c15e9a0 2024-04-15T15:19:59.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.784+0000 7f7682a00640 1 -- 172.21.15.49:0/1717202766 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f768401a070 con 0x7f768c15e9a0 2024-04-15T15:19:59.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.785+0000 7f7682a00640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f766806c8f0 0x7f766806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:19:59.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.785+0000 7f7682a00640 1 -- 172.21.15.49:0/1717202766 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5609+0+0 (secure 0 0 0) 0x7f768408be50 con 0x7f768c15e9a0 2024-04-15T15:19:59.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.785+0000 7f768a800640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f766806c8f0 0x7f766806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:19:59.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.786+0000 7f768a800640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f766806c8f0 0x7f766806edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f767c035000 tx=0x7f767c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:19:59.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:19:59.790+0000 7f7682a00640 1 -- 172.21.15.49:0/1717202766 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7684059bf0 con 0x7f768c15e9a0 2024-04-15T15:20:00.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.035+0000 7f7690a00640 1 -- 172.21.15.49:0/1717202766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f768c068ef0 con 0x7f768c15e9a0 2024-04-15T15:20:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.592+0000 7f7682a00640 1 -- 172.21.15.49:0/1717202766 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f768405def0 con 0x7f768c15e9a0 2024-04-15T15:20:00.597 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 4 and data pool 5 2024-04-15T15:20:00.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.604+0000 7f7690a00640 1 -- 172.21.15.49:0/1717202766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f766806c8f0 msgr2=0x7f766806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.604+0000 7f7690a00640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f766806c8f0 0x7f766806edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f767c035000 tx=0x7f767c030000 comp rx=0 tx=0).stop 2024-04-15T15:20:00.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.604+0000 7f7690a00640 1 -- 172.21.15.49:0/1717202766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f768c15e9a0 msgr2=0x7f768c13fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.604+0000 7f7690a00640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f768c15e9a0 0x7f768c13fc90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f768400e880 tx=0x7f768400ed40 comp rx=0 tx=0).stop 2024-04-15T15:20:00.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.605+0000 7f7690a00640 1 -- 172.21.15.49:0/1717202766 shutdown_connections 2024-04-15T15:20:00.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.605+0000 7f7690a00640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f766806c8f0 0x7f766806edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.605+0000 7f7690a00640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f768c15e9a0 0x7f768c13fc90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.605+0000 7f7690a00640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768c0cfc00 0x7f768c140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.605+0000 7f7690a00640 1 --2- 172.21.15.49:0/1717202766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f768c0ba3e0 0x7f768c1618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.606+0000 7f7690a00640 1 -- 172.21.15.49:0/1717202766 >> 172.21.15.49:0/1717202766 conn(0x7f768c06ce40 msgr2=0x7f768c0d3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.606+0000 7f7690a00640 1 -- 172.21.15.49:0/1717202766 shutdown_connections 2024-04-15T15:20:00.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.607+0000 7f7690a00640 1 -- 172.21.15.49:0/1717202766 wait complete. 2024-04-15T15:20:00.645 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:00.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.835+0000 7f11b1e00640 1 Processor -- start 2024-04-15T15:20:00.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.835+0000 7f11b1e00640 1 -- start start 2024-04-15T15:20:00.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.836+0000 7f11b1e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f11ac1481f0 0x7f11ac15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.836+0000 7f11b1e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11ac0728f0 0x7f11ac1601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.836+0000 7f11b1e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11ac1485d0 con 0x7f11ac0affc0 2024-04-15T15:20:00.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.836+0000 7f11b1e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f11ac15ffc0 con 0x7f11ac1481f0 2024-04-15T15:20:00.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.836+0000 7f11b1e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f11ac0aef20 con 0x7f11ac0728f0 2024-04-15T15:20:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.837+0000 7f11aaa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f11ac1481f0 0x7f11ac15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.837+0000 7f11aaa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f11ac1481f0 0x7f11ac15fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50018/0 (socket says 172.21.15.49:50018) 2024-04-15T15:20:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.837+0000 7f11aaa00640 1 -- 172.21.15.49:0/631961799 learned_addr learned my addr 172.21.15.49:0/631961799 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.837+0000 7f11ab400640 1 --2- 172.21.15.49:0/631961799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11ac0728f0 0x7f11ac1601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.837+0000 7f11aaa00640 1 -- 172.21.15.49:0/631961799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11ac0728f0 msgr2=0x7f11ac1601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.837+0000 7f11aaa00640 1 --2- 172.21.15.49:0/631961799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11ac0728f0 0x7f11ac1601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.837+0000 7f11aaa00640 1 -- 172.21.15.49:0/631961799 >> v1:172.21.15.107:6789/0 conn(0x7f11ac0affc0 legacy=0x7f11ac0721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:00.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.837+0000 7f11aaa00640 1 -- 172.21.15.49:0/631961799 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac0af200 con 0x7f11ac1481f0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.839+0000 7f11aaa00640 1 --2- 172.21.15.49:0/631961799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac1481f0 0x7f11ac15fa80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f119c009870 tx=0x7f119c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc006a184e6ca6bd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.839+0000 7f11a9600640 1 -- 172.21.15.49:0/631961799 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c035020 con 0x7f11ac1481f0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.839+0000 7f11a9600640 1 -- 172.21.15.49:0/631961799 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c031640 con 0x7f11ac1481f0 2024-04-15T15:20:00.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.839+0000 7f11a9600640 1 -- 172.21.15.49:0/631961799 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c02d040 con 0x7f11ac1481f0 2024-04-15T15:20:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.840+0000 7f11b1e00640 1 -- 172.21.15.49:0/631961799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac1481f0 msgr2=0x7f11ac15fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.840+0000 7f11b1e00640 1 --2- 172.21.15.49:0/631961799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac1481f0 0x7f11ac15fa80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f119c009870 tx=0x7f119c02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.840+0000 7f11b1e00640 1 -- 172.21.15.49:0/631961799 shutdown_connections 2024-04-15T15:20:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.840+0000 7f11b1e00640 1 --2- 172.21.15.49:0/631961799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11ac0728f0 0x7f11ac1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.840+0000 7f11b1e00640 1 --2- 172.21.15.49:0/631961799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac1481f0 0x7f11ac15fa80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.840+0000 7f11b1e00640 1 -- 172.21.15.49:0/631961799 >> 172.21.15.49:0/631961799 conn(0x7f11ac0b27b0 msgr2=0x7f11ac0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.840+0000 7f11b1e00640 1 -- 172.21.15.49:0/631961799 shutdown_connections 2024-04-15T15:20:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.840+0000 7f11b1e00640 1 -- 172.21.15.49:0/631961799 wait complete. 2024-04-15T15:20:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.842+0000 7f11b1e00640 1 Processor -- start 2024-04-15T15:20:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.842+0000 7f11b1e00640 1 -- start start 2024-04-15T15:20:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.843+0000 7f11b1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac0728f0 0x7f11ac13a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.843+0000 7f11b1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11ac0affc0 0x7f11ac152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.843+0000 7f11b1e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11ac1481f0 0x7f11ac13cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.843+0000 7f11b1e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac0afaa0 con 0x7f11ac0728f0 2024-04-15T15:20:00.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.843+0000 7f11b1e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac0af7c0 con 0x7f11ac0affc0 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.843+0000 7f11b1e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f11ac0b5560 con 0x7f11ac1481f0 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.843+0000 7f11aaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac0728f0 0x7f11ac13a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.843+0000 7f11aaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac0728f0 0x7f11ac13a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50020/0 (socket says 172.21.15.49:50020) 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.843+0000 7f11aaa00640 1 -- 172.21.15.49:0/461098881 learned_addr learned my addr 172.21.15.49:0/461098881 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.844+0000 7f11ab400640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11ac1481f0 0x7f11ac13cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.844+0000 7f11aa000640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11ac0affc0 0x7f11ac152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.844+0000 7f11aaa00640 1 -- 172.21.15.49:0/461098881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11ac1481f0 msgr2=0x7f11ac13cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.844+0000 7f11aaa00640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11ac1481f0 0x7f11ac13cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.844+0000 7f11aaa00640 1 -- 172.21.15.49:0/461098881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11ac0affc0 msgr2=0x7f11ac152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.844+0000 7f11aaa00640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11ac0affc0 0x7f11ac152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:00.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.844+0000 7f11aaa00640 1 -- 172.21.15.49:0/461098881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac13b0e0 con 0x7f11ac0728f0 2024-04-15T15:20:00.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.844+0000 7f11aaa00640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac0728f0 0x7f11ac13a970 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f119c02b930 tx=0x7f119c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.845+0000 7f11a3400640 1 -- 172.21.15.49:0/461098881 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c040070 con 0x7f11ac0728f0 2024-04-15T15:20:00.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.845+0000 7f11a3400640 1 -- 172.21.15.49:0/461098881 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c0399f0 con 0x7f11ac0728f0 2024-04-15T15:20:00.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.845+0000 7f11a3400640 1 -- 172.21.15.49:0/461098881 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c035040 con 0x7f11ac0728f0 2024-04-15T15:20:00.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.845+0000 7f11b1e00640 1 -- 172.21.15.49:0/461098881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac13b3c0 con 0x7f11ac0728f0 2024-04-15T15:20:00.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.845+0000 7f11b1e00640 1 -- 172.21.15.49:0/461098881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac13b980 con 0x7f11ac0728f0 2024-04-15T15:20:00.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.846+0000 7f11b1e00640 1 -- 172.21.15.49:0/461098881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ac068c80 con 0x7f11ac0728f0 2024-04-15T15:20:00.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.847+0000 7f11a3400640 1 -- 172.21.15.49:0/461098881 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f119c04b060 con 0x7f11ac0728f0 2024-04-15T15:20:00.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.848+0000 7f11a3400640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f118c06c920 0x7f118c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:00.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.848+0000 7f11a3400640 1 -- 172.21.15.49:0/461098881 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f119c0ad430 con 0x7f11ac0728f0 2024-04-15T15:20:00.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.848+0000 7f11aa000640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f118c06c920 0x7f118c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:00.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.849+0000 7f11aa000640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f118c06c920 0x7f118c06ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0cfea0 tx=0x7f1198009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:00.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:00.853+0000 7f11a3400640 1 -- 172.21.15.49:0/461098881 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f119c07b850 con 0x7f11ac0728f0 2024-04-15T15:20:01.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.057+0000 7f11b1e00640 1 -- 172.21.15.49:0/461098881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11ac160b90 con 0x7f11ac0728f0 2024-04-15T15:20:01.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.059+0000 7f11a3400640 1 -- 172.21.15.49:0/461098881 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16682 (secure 0 0 0) 0x7f119c07b150 con 0x7f11ac0728f0 2024-04-15T15:20:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:01.061 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":34,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:00.037520+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.062+0000 7f11b1e00640 1 -- 172.21.15.49:0/461098881 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f118c06c920 msgr2=0x7f118c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.062+0000 7f11b1e00640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f118c06c920 0x7f118c06ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0cfea0 tx=0x7f1198009040 comp rx=0 tx=0).stop 2024-04-15T15:20:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.062+0000 7f11b1e00640 1 -- 172.21.15.49:0/461098881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac0728f0 msgr2=0x7f11ac13a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.062+0000 7f11b1e00640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac0728f0 0x7f11ac13a970 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f119c02b930 tx=0x7f119c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.063+0000 7f11b1e00640 1 -- 172.21.15.49:0/461098881 shutdown_connections 2024-04-15T15:20:01.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.063+0000 7f11b1e00640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f118c06c920 0x7f118c06ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.063+0000 7f11b1e00640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11ac1481f0 0x7f11ac13cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.063+0000 7f11b1e00640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11ac0affc0 0x7f11ac152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.063+0000 7f11b1e00640 1 --2- 172.21.15.49:0/461098881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11ac0728f0 0x7f11ac13a970 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.063+0000 7f11b1e00640 1 -- 172.21.15.49:0/461098881 >> 172.21.15.49:0/461098881 conn(0x7f11ac0b27b0 msgr2=0x7f11ac0b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:01.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.063+0000 7f11b1e00640 1 -- 172.21.15.49:0/461098881 shutdown_connections 2024-04-15T15:20:01.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.063+0000 7f11b1e00640 1 -- 172.21.15.49:0/461098881 wait complete. 2024-04-15T15:20:01.085 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.301+0000 7f843fe00640 1 Processor -- start 2024-04-15T15:20:01.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.302+0000 7f843fe00640 1 -- start start 2024-04-15T15:20:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.303+0000 7f843fe00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f843815b6d0 0x7f84380cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.303+0000 7f843fe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8438070210 0x7f84380b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.303+0000 7f843fe00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f843806f940 con 0x7f84380ce250 2024-04-15T15:20:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.303+0000 7f843fe00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f843806e8a0 con 0x7f8438070210 2024-04-15T15:20:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.303+0000 7f843fe00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f843806eb80 con 0x7f843815b6d0 2024-04-15T15:20:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.303+0000 7f843ca00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8438070210 0x7f84380b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.303+0000 7f843de00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f84380ce250 0x7f84380721e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:52090/0 (socket says 172.21.15.49:52090) 2024-04-15T15:20:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.303+0000 7f843de00640 1 -- 172.21.15.49:0/2332487234 learned_addr learned my addr 172.21.15.49:0/2332487234 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.304+0000 7f843d400640 1 --2- 172.21.15.49:0/2332487234 >> v2:172.21.15.107:3300/0 conn(0x7f843815b6d0 0x7f84380cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.304+0000 7f843ca00640 1 -- 172.21.15.49:0/2332487234 >> v2:172.21.15.107:3300/0 conn(0x7f843815b6d0 msgr2=0x7f84380cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.304+0000 7f843ca00640 1 --2- 172.21.15.49:0/2332487234 >> v2:172.21.15.107:3300/0 conn(0x7f843815b6d0 0x7f84380cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.304+0000 7f843ca00640 1 -- 172.21.15.49:0/2332487234 >> v1:172.21.15.49:6789/0 conn(0x7f84380ce250 legacy=0x7f84380721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.304+0000 7f843ca00640 1 -- 172.21.15.49:0/2332487234 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843806ee60 con 0x7f8438070210 2024-04-15T15:20:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.304+0000 7f8437e00640 1 -- 172.21.15.49:0/2332487234 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 772210344 0 0) 0x7f843806f940 con 0x7f84380ce250 2024-04-15T15:20:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.304+0000 7f843d400640 1 --2- 172.21.15.49:0/2332487234 >> v2:172.21.15.107:3300/0 conn(0x7f843815b6d0 0x7f84380cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:01.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.305+0000 7f843ca00640 1 --2- 172.21.15.49:0/2332487234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 0x7f84380b4ae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8424009d90 tx=0x7f842402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac2dc1c52f74760 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f8437e00640 1 -- 172.21.15.49:0/2332487234 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424039070 con 0x7f8438070210 2024-04-15T15:20:01.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f8437e00640 1 -- 172.21.15.49:0/2332487234 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842402cbd0 con 0x7f8438070210 2024-04-15T15:20:01.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f8437e00640 1 -- 172.21.15.49:0/2332487234 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424034440 con 0x7f8438070210 2024-04-15T15:20:01.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f843fe00640 1 -- 172.21.15.49:0/2332487234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 msgr2=0x7f84380b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f843fe00640 1 --2- 172.21.15.49:0/2332487234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 0x7f84380b4ae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8424009d90 tx=0x7f842402be20 comp rx=0 tx=0).stop 2024-04-15T15:20:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f843fe00640 1 -- 172.21.15.49:0/2332487234 shutdown_connections 2024-04-15T15:20:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f843fe00640 1 --2- 172.21.15.49:0/2332487234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 0x7f84380b4ae0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f843fe00640 1 --2- 172.21.15.49:0/2332487234 >> v2:172.21.15.107:3300/0 conn(0x7f843815b6d0 0x7f84380cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f843fe00640 1 -- 172.21.15.49:0/2332487234 >> 172.21.15.49:0/2332487234 conn(0x7f84380b27b0 msgr2=0x7f84380b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f843fe00640 1 -- 172.21.15.49:0/2332487234 shutdown_connections 2024-04-15T15:20:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.306+0000 7f843fe00640 1 -- 172.21.15.49:0/2332487234 wait complete. 2024-04-15T15:20:01.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.308+0000 7f843fe00640 1 Processor -- start 2024-04-15T15:20:01.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.308+0000 7f843fe00640 1 -- start start 2024-04-15T15:20:01.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.309+0000 7f843fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 0x7f84380af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.309+0000 7f843fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84380ce250 0x7f8438151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.309+0000 7f843fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f843815b6d0 0x7f84380b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.309+0000 7f843fe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f843815cac0 con 0x7f8438070210 2024-04-15T15:20:01.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.309+0000 7f843fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f843806f420 con 0x7f843815b6d0 2024-04-15T15:20:01.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.309+0000 7f843fe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f843815cc40 con 0x7f84380ce250 2024-04-15T15:20:01.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.309+0000 7f843d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 0x7f84380af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.309+0000 7f843d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 0x7f84380af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50034/0 (socket says 172.21.15.49:50034) 2024-04-15T15:20:01.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843d400640 1 -- 172.21.15.49:0/824413291 learned_addr learned my addr 172.21.15.49:0/824413291 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:01.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843ca00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84380ce250 0x7f8438151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843de00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f843815b6d0 0x7f84380b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843d400640 1 -- 172.21.15.49:0/824413291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84380ce250 msgr2=0x7f8438151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843d400640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84380ce250 0x7f8438151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843d400640 1 -- 172.21.15.49:0/824413291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f843815b6d0 msgr2=0x7f84380b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843d400640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f843815b6d0 0x7f84380b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843d400640 1 -- 172.21.15.49:0/824413291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8438153350 con 0x7f8438070210 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843ca00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84380ce250 0x7f8438151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843de00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f843815b6d0 0x7f84380b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.310+0000 7f843d400640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 0x7f84380af590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f842800c820 tx=0x7f842800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.311+0000 7f8435600640 1 -- 172.21.15.49:0/824413291 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428015070 con 0x7f8438070210 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.311+0000 7f8435600640 1 -- 172.21.15.49:0/824413291 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8428007a30 con 0x7f8438070210 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.311+0000 7f8435600640 1 -- 172.21.15.49:0/824413291 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428007de0 con 0x7f8438070210 2024-04-15T15:20:01.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.311+0000 7f843fe00640 1 -- 172.21.15.49:0/824413291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84380afad0 con 0x7f8438070210 2024-04-15T15:20:01.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.311+0000 7f843fe00640 1 -- 172.21.15.49:0/824413291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84380aeab0 con 0x7f8438070210 2024-04-15T15:20:01.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.312+0000 7f843fe00640 1 -- 172.21.15.49:0/824413291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8438068c80 con 0x7f8438070210 2024-04-15T15:20:01.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.313+0000 7f8435600640 1 -- 172.21.15.49:0/824413291 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8428018070 con 0x7f8438070210 2024-04-15T15:20:01.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.314+0000 7f8435600640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f841806c8f0 0x7f841806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.314+0000 7f8435600640 1 -- 172.21.15.49:0/824413291 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f842808ba00 con 0x7f8438070210 2024-04-15T15:20:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.314+0000 7f843ca00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f841806c8f0 0x7f841806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.315+0000 7f843ca00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f841806c8f0 0x7f841806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8424036000 tx=0x7f8424030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.319+0000 7f8435600640 1 -- 172.21.15.49:0/824413291 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8428059e20 con 0x7f8438070210 2024-04-15T15:20:01.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.515+0000 7f843fe00640 1 -- 172.21.15.49:0/824413291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84380d0560 con 0x7f8438070210 2024-04-15T15:20:01.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.516+0000 7f8435600640 1 -- 172.21.15.49:0/824413291 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16682 (secure 0 0 0) 0x7f8428059720 con 0x7f8438070210 2024-04-15T15:20:01.518 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:01.518 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":34,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:00.037520+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 -- 172.21.15.49:0/824413291 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f841806c8f0 msgr2=0x7f841806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f841806c8f0 0x7f841806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8424036000 tx=0x7f8424030000 comp rx=0 tx=0).stop 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 -- 172.21.15.49:0/824413291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 msgr2=0x7f84380af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 0x7f84380af590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f842800c820 tx=0x7f842800cce0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 -- 172.21.15.49:0/824413291 shutdown_connections 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f841806c8f0 0x7f841806edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84380ce250 0x7f8438151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f843815b6d0 0x7f84380b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 --2- 172.21.15.49:0/824413291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438070210 0x7f84380af590 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 -- 172.21.15.49:0/824413291 >> 172.21.15.49:0/824413291 conn(0x7f84380b27b0 msgr2=0x7f8438158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:01.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 -- 172.21.15.49:0/824413291 shutdown_connections 2024-04-15T15:20:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.519+0000 7f843fe00640 1 -- 172.21.15.49:0/824413291 wait complete. 2024-04-15T15:20:01.538 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-15T15:20:01.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.761+0000 7f8c6ea00640 1 Processor -- start 2024-04-15T15:20:01.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.761+0000 7f8c6ea00640 1 -- start start 2024-04-15T15:20:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.763+0000 7f8c6ea00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8c681461c0 0x7f8c68071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.763+0000 7f8c6ea00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c680cd2d0 con 0x7f8c680cebb0 2024-04-15T15:20:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.763+0000 7f8c6ea00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c680cd5b0 con 0x7f8c68072420 2024-04-15T15:20:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.763+0000 7f8c6ea00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f8c680cf1d0 con 0x7f8c681461c0 2024-04-15T15:20:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.763+0000 7f8c6d600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8c681461c0 0x7f8c68071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.763+0000 7f8c6e000640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8c680cebb0 0x7f8c6806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:52102/0 (socket says 172.21.15.49:52102) 2024-04-15T15:20:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.763+0000 7f8c6e000640 1 -- 172.21.15.49:0/432608061 learned_addr learned my addr 172.21.15.49:0/432608061 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.764+0000 7f8c67e00640 1 -- 172.21.15.49:0/432608061 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2423304291 0 0) 0x7f8c680cd2d0 con 0x7f8c680cebb0 2024-04-15T15:20:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.764+0000 7f8c67e00640 1 -- 172.21.15.49:0/432608061 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c50003520 con 0x7f8c680cebb0 2024-04-15T15:20:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.764+0000 7f8c6d600640 1 -- 172.21.15.49:0/432608061 >> v1:172.21.15.107:6789/0 conn(0x7f8c68072420 legacy=0x7f8c680ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.764+0000 7f8c6d600640 1 -- 172.21.15.49:0/432608061 >> v1:172.21.15.49:6789/0 conn(0x7f8c680cebb0 legacy=0x7f8c6806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.764+0000 7f8c6d600640 1 -- 172.21.15.49:0/432608061 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c680cd890 con 0x7f8c681461c0 2024-04-15T15:20:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.766+0000 7f8c6d600640 1 --2- 172.21.15.49:0/432608061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c681461c0 0x7f8c68071ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0097a0 tx=0x7f8c5c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc6a4fb77dbd626 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.767+0000 7f8c67e00640 1 -- 172.21.15.49:0/432608061 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c034070 con 0x7f8c681461c0 2024-04-15T15:20:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.767+0000 7f8c67e00640 1 -- 172.21.15.49:0/432608061 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5c030610 con 0x7f8c681461c0 2024-04-15T15:20:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.767+0000 7f8c67e00640 1 -- 172.21.15.49:0/432608061 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0309c0 con 0x7f8c681461c0 2024-04-15T15:20:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.767+0000 7f8c6ea00640 1 -- 172.21.15.49:0/432608061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c681461c0 msgr2=0x7f8c68071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.767+0000 7f8c6ea00640 1 --2- 172.21.15.49:0/432608061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c681461c0 0x7f8c68071ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0097a0 tx=0x7f8c5c02b960 comp rx=0 tx=0).stop 2024-04-15T15:20:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.767+0000 7f8c6ea00640 1 -- 172.21.15.49:0/432608061 shutdown_connections 2024-04-15T15:20:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.767+0000 7f8c6ea00640 1 --2- 172.21.15.49:0/432608061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c681461c0 0x7f8c68071ee0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.767+0000 7f8c6ea00640 1 -- 172.21.15.49:0/432608061 >> 172.21.15.49:0/432608061 conn(0x7f8c6806b930 msgr2=0x7f8c6806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.768+0000 7f8c6ea00640 1 -- 172.21.15.49:0/432608061 shutdown_connections 2024-04-15T15:20:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.768+0000 7f8c6ea00640 1 -- 172.21.15.49:0/432608061 wait complete. 2024-04-15T15:20:01.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.770+0000 7f8c6ea00640 1 Processor -- start 2024-04-15T15:20:01.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.770+0000 7f8c6ea00640 1 -- start start 2024-04-15T15:20:01.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.771+0000 7f8c6ea00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c68072420 0x7f8c680abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.771+0000 7f8c6ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c680cebb0 0x7f8c680ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.771+0000 7f8c6ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c681461c0 0x7f8c680acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.771+0000 7f8c6ea00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8c680cd080 con 0x7f8c680cebb0 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.771+0000 7f8c6ea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6806e630 con 0x7f8c681461c0 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.771+0000 7f8c6ea00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f8c6806e7b0 con 0x7f8c68072420 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.771+0000 7f8c6cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c680cebb0 0x7f8c680ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.772+0000 7f8c6cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c680cebb0 0x7f8c680ac5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50050/0 (socket says 172.21.15.49:50050) 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.772+0000 7f8c6cc00640 1 -- 172.21.15.49:0/3467441054 learned_addr learned my addr 172.21.15.49:0/3467441054 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.772+0000 7f8c6e000640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c681461c0 0x7f8c680acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.772+0000 7f8c6cc00640 1 -- 172.21.15.49:0/3467441054 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c68072420 msgr2=0x7f8c680abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.772+0000 7f8c6cc00640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c68072420 0x7f8c680abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.772+0000 7f8c6cc00640 1 -- 172.21.15.49:0/3467441054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c681461c0 msgr2=0x7f8c680acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.772+0000 7f8c6cc00640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c681461c0 0x7f8c680acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:01.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.772+0000 7f8c6cc00640 1 -- 172.21.15.49:0/3467441054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c680adc90 con 0x7f8c680cebb0 2024-04-15T15:20:01.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.772+0000 7f8c6e000640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c681461c0 0x7f8c680acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:01.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.772+0000 7f8c6cc00640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c680cebb0 0x7f8c680ac5e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c400 tx=0x7f8c5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.773+0000 7f8c65600640 1 -- 172.21.15.49:0/3467441054 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c540052d0 con 0x7f8c680cebb0 2024-04-15T15:20:01.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.773+0000 7f8c65600640 1 -- 172.21.15.49:0/3467441054 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c54005490 con 0x7f8c680cebb0 2024-04-15T15:20:01.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.773+0000 7f8c65600640 1 -- 172.21.15.49:0/3467441054 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54007260 con 0x7f8c680cebb0 2024-04-15T15:20:01.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.773+0000 7f8c6ea00640 1 -- 172.21.15.49:0/3467441054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c680ac190 con 0x7f8c680cebb0 2024-04-15T15:20:01.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.773+0000 7f8c6ea00640 1 -- 172.21.15.49:0/3467441054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c680ae210 con 0x7f8c680cebb0 2024-04-15T15:20:01.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.774+0000 7f8c6ea00640 1 -- 172.21.15.49:0/3467441054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c680755c0 con 0x7f8c680cebb0 2024-04-15T15:20:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.775+0000 7f8c65600640 1 -- 172.21.15.49:0/3467441054 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c54007400 con 0x7f8c680cebb0 2024-04-15T15:20:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.776+0000 7f8c65600640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8c4806c8f0 0x7f8c4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.776+0000 7f8c65600640 1 -- 172.21.15.49:0/3467441054 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8c5408b0e0 con 0x7f8c680cebb0 2024-04-15T15:20:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.776+0000 7f8c6d600640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8c4806c8f0 0x7f8c4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:01.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.777+0000 7f8c6d600640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8c4806c8f0 0x7f8c4806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c036000 tx=0x7f8c5c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:01.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:01.782+0000 7f8c65600640 1 -- 172.21.15.49:0/3467441054 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c54059510 con 0x7f8c680cebb0 2024-04-15T15:20:02.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.021+0000 7f8c6ea00640 1 -- 172.21.15.49:0/3467441054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8c680ac3c0 con 0x7f8c680cebb0 2024-04-15T15:20:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.602+0000 7f8c65600640 1 -- 172.21.15.49:0/3467441054 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f8c54058e10 con 0x7f8c680cebb0 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.606+0000 7f8c6ea00640 1 -- 172.21.15.49:0/3467441054 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8c4806c8f0 msgr2=0x7f8c4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.606+0000 7f8c6ea00640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8c4806c8f0 0x7f8c4806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c036000 tx=0x7f8c5c030000 comp rx=0 tx=0).stop 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.606+0000 7f8c6ea00640 1 -- 172.21.15.49:0/3467441054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c680cebb0 msgr2=0x7f8c680ac5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.606+0000 7f8c6ea00640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c680cebb0 0x7f8c680ac5e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c400 tx=0x7f8c5400c8c0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.606+0000 7f8c6ea00640 1 -- 172.21.15.49:0/3467441054 shutdown_connections 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.606+0000 7f8c6ea00640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8c4806c8f0 0x7f8c4806edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.606+0000 7f8c6ea00640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c681461c0 0x7f8c680acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.606+0000 7f8c6ea00640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c680cebb0 0x7f8c680ac5e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.606+0000 7f8c6ea00640 1 --2- 172.21.15.49:0/3467441054 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c68072420 0x7f8c680abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.606+0000 7f8c6ea00640 1 -- 172.21.15.49:0/3467441054 >> 172.21.15.49:0/3467441054 conn(0x7f8c6806b930 msgr2=0x7f8c6806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.607+0000 7f8c6ea00640 1 -- 172.21.15.49:0/3467441054 shutdown_connections 2024-04-15T15:20:02.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.607+0000 7f8c6ea00640 1 -- 172.21.15.49:0/3467441054 wait complete. 2024-04-15T15:20:02.635 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-15T15:20:02.635 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-15T15:20:02.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.869+0000 7f321cc00640 1 Processor -- start 2024-04-15T15:20:02.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.869+0000 7f321cc00640 1 -- start start 2024-04-15T15:20:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.870+0000 7f321cc00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218145070 0x7f3218072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.870+0000 7f321cc00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3218072690 0x7f321815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.870+0000 7f321cc00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321815ffe0 con 0x7f3218163520 2024-04-15T15:20:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.870+0000 7f321cc00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f32180afde0 con 0x7f3218072690 2024-04-15T15:20:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.870+0000 7f321cc00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f32181627f0 con 0x7f3218145070 2024-04-15T15:20:02.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.871+0000 7f320fe00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3218163520 0x7f321815f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:52110/0 (socket says 172.21.15.49:52110) 2024-04-15T15:20:02.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.871+0000 7f320fe00640 1 -- 172.21.15.49:0/2652174198 learned_addr learned my addr 172.21.15.49:0/2652174198 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:02.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.871+0000 7f3216800640 1 --2- 172.21.15.49:0/2652174198 >> v2:172.21.15.107:3300/0 conn(0x7f3218072690 0x7f321815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.871+0000 7f3215e00640 1 --2- 172.21.15.49:0/2652174198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218145070 0x7f3218072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.872+0000 7f3215400640 1 -- 172.21.15.49:0/2652174198 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3650001918 0 0) 0x7f321815ffe0 con 0x7f3218163520 2024-04-15T15:20:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.872+0000 7f3215400640 1 -- 172.21.15.49:0/2652174198 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31fc003520 con 0x7f3218163520 2024-04-15T15:20:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.872+0000 7f3216800640 1 -- 172.21.15.49:0/2652174198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218145070 msgr2=0x7f3218072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.872+0000 7f3216800640 1 --2- 172.21.15.49:0/2652174198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218145070 0x7f3218072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.872+0000 7f3216800640 1 -- 172.21.15.49:0/2652174198 >> v1:172.21.15.49:6789/0 conn(0x7f3218163520 legacy=0x7f321815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.872+0000 7f3216800640 1 -- 172.21.15.49:0/2652174198 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3218162a40 con 0x7f3218072690 2024-04-15T15:20:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.872+0000 7f3215e00640 1 --2- 172.21.15.49:0/2652174198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218145070 0x7f3218072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.874+0000 7f3216800640 1 --2- 172.21.15.49:0/2652174198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3218072690 0x7f321815f8b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3210009d90 tx=0x7f321002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d58f744a2413fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.874+0000 7f3215400640 1 -- 172.21.15.49:0/2652174198 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210039070 con 0x7f3218072690 2024-04-15T15:20:02.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.874+0000 7f3215400640 1 -- 172.21.15.49:0/2652174198 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321002cbc0 con 0x7f3218072690 2024-04-15T15:20:02.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.874+0000 7f3215400640 1 -- 172.21.15.49:0/2652174198 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210034440 con 0x7f3218072690 2024-04-15T15:20:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.875+0000 7f321cc00640 1 -- 172.21.15.49:0/2652174198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3218072690 msgr2=0x7f321815f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.875+0000 7f321cc00640 1 --2- 172.21.15.49:0/2652174198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3218072690 0x7f321815f8b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3210009d90 tx=0x7f321002be10 comp rx=0 tx=0).stop 2024-04-15T15:20:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.875+0000 7f321cc00640 1 -- 172.21.15.49:0/2652174198 shutdown_connections 2024-04-15T15:20:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.875+0000 7f321cc00640 1 --2- 172.21.15.49:0/2652174198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3218072690 0x7f321815f8b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.875+0000 7f321cc00640 1 --2- 172.21.15.49:0/2652174198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218145070 0x7f3218072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.875+0000 7f321cc00640 1 -- 172.21.15.49:0/2652174198 >> 172.21.15.49:0/2652174198 conn(0x7f32180b2460 msgr2=0x7f32180b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.875+0000 7f321cc00640 1 -- 172.21.15.49:0/2652174198 shutdown_connections 2024-04-15T15:20:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.875+0000 7f321cc00640 1 -- 172.21.15.49:0/2652174198 wait complete. 2024-04-15T15:20:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.878+0000 7f321cc00640 1 Processor -- start 2024-04-15T15:20:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.878+0000 7f321cc00640 1 -- start start 2024-04-15T15:20:02.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.879+0000 7f321cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3218072690 0x7f32181455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.879+0000 7f321cc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3218145070 0x7f321815dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.879+0000 7f321cc00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218163520 0x7f321815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.879+0000 7f321cc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3218145450 con 0x7f3218145070 2024-04-15T15:20:02.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.879+0000 7f321cc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3218163000 con 0x7f3218072690 2024-04-15T15:20:02.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.879+0000 7f321cc00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f32180aedd0 con 0x7f3218163520 2024-04-15T15:20:02.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.879+0000 7f320fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3218145070 0x7f321815dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.879+0000 7f320fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3218145070 0x7f321815dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50064/0 (socket says 172.21.15.49:50064) 2024-04-15T15:20:02.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.879+0000 7f320fe00640 1 -- 172.21.15.49:0/1854377729 learned_addr learned my addr 172.21.15.49:0/1854377729 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:02.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.880+0000 7f3215e00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3218072690 0x7f32181455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.880+0000 7f3216800640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218163520 0x7f321815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.880+0000 7f320fe00640 1 -- 172.21.15.49:0/1854377729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218163520 msgr2=0x7f321815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.880+0000 7f320fe00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218163520 0x7f321815c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.880+0000 7f320fe00640 1 -- 172.21.15.49:0/1854377729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3218072690 msgr2=0x7f32181455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:02.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.880+0000 7f320fe00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3218072690 0x7f32181455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:02.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.880+0000 7f320fe00640 1 -- 172.21.15.49:0/1854377729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3218145cd0 con 0x7f3218145070 2024-04-15T15:20:02.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.880+0000 7f320fe00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3218145070 0x7f321815dc70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f320000c410 tx=0x7f320000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.881+0000 7f320e000640 1 -- 172.21.15.49:0/1854377729 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200017070 con 0x7f3218145070 2024-04-15T15:20:02.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.881+0000 7f320e000640 1 -- 172.21.15.49:0/1854377729 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3200005ce0 con 0x7f3218145070 2024-04-15T15:20:02.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.881+0000 7f320e000640 1 -- 172.21.15.49:0/1854377729 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32000072f0 con 0x7f3218145070 2024-04-15T15:20:02.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.881+0000 7f321cc00640 1 -- 172.21.15.49:0/1854377729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3218145fb0 con 0x7f3218145070 2024-04-15T15:20:02.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.881+0000 7f321cc00640 1 -- 172.21.15.49:0/1854377729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3218146570 con 0x7f3218145070 2024-04-15T15:20:02.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.882+0000 7f321cc00640 1 -- 172.21.15.49:0/1854377729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3218068c80 con 0x7f3218145070 2024-04-15T15:20:02.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.883+0000 7f320e000640 1 -- 172.21.15.49:0/1854377729 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f320001a070 con 0x7f3218145070 2024-04-15T15:20:02.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.884+0000 7f320e000640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f406c8f0 0x7f31f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:02.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.884+0000 7f320e000640 1 -- 172.21.15.49:0/1854377729 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f320008b070 con 0x7f3218145070 2024-04-15T15:20:02.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.885+0000 7f3215e00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f406c8f0 0x7f31f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:02.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.886+0000 7f3215e00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f406c8f0 0x7f31f406edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3208009560 tx=0x7f3208008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:02.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:02.889+0000 7f320e000640 1 -- 172.21.15.49:0/1854377729 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32000594c0 con 0x7f3218145070 2024-04-15T15:20:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.149+0000 7f321cc00640 1 -- 172.21.15.49:0/1854377729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f321806f8d0 con 0x7f3218145070 2024-04-15T15:20:03.307 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-04-15T15:20:03.305+0000 7f12f2000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:20:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.612+0000 7f320e000640 1 -- 172.21.15.49:0/1854377729 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f3200058dc0 con 0x7f3218145070 2024-04-15T15:20:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 -- 172.21.15.49:0/1854377729 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f406c8f0 msgr2=0x7f31f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f406c8f0 0x7f31f406edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3208009560 tx=0x7f3208008040 comp rx=0 tx=0).stop 2024-04-15T15:20:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 -- 172.21.15.49:0/1854377729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3218145070 msgr2=0x7f321815dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3218145070 0x7f321815dc70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f320000c410 tx=0x7f320000c8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 -- 172.21.15.49:0/1854377729 shutdown_connections 2024-04-15T15:20:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f406c8f0 0x7f31f406edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3218163520 0x7f321815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3218145070 0x7f321815dc70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 --2- 172.21.15.49:0/1854377729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3218072690 0x7f32181455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 -- 172.21.15.49:0/1854377729 >> 172.21.15.49:0/1854377729 conn(0x7f32180b2460 msgr2=0x7f32180af230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:03.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 -- 172.21.15.49:0/1854377729 shutdown_connections 2024-04-15T15:20:03.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.616+0000 7f321cc00640 1 -- 172.21.15.49:0/1854377729 wait complete. 2024-04-15T15:20:03.643 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-15T15:20:03.643 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:03.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.872+0000 7f191d200640 1 Processor -- start 2024-04-15T15:20:03.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.872+0000 7f191d200640 1 -- start start 2024-04-15T15:20:03.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.873+0000 7f191d200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1918162020 0x7f1918162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.873+0000 7f191d200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19180d1c90 0x7f19180d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.873+0000 7f191d200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19180d0090 con 0x7f19180cf110 2024-04-15T15:20:03.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.873+0000 7f191d200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f1918163440 con 0x7f19180d1c90 2024-04-15T15:20:03.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.873+0000 7f191d200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f19180b7b30 con 0x7f1918162020 2024-04-15T15:20:03.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.874+0000 7f1917e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19180d1c90 0x7f19180d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.874+0000 7f1917e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19180d1c90 0x7f19180d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50076/0 (socket says 172.21.15.49:50076) 2024-04-15T15:20:03.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.874+0000 7f1917e00640 1 -- 172.21.15.49:0/4149082255 learned_addr learned my addr 172.21.15.49:0/4149082255 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:03.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.874+0000 7f1917400640 1 --2- 172.21.15.49:0/4149082255 >> v2:172.21.15.107:3300/0 conn(0x7f1918162020 0x7f1918162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.875+0000 7f1917e00640 1 -- 172.21.15.49:0/4149082255 >> v2:172.21.15.107:3300/0 conn(0x7f1918162020 msgr2=0x7f1918162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:03.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.875+0000 7f1917e00640 1 --2- 172.21.15.49:0/4149082255 >> v2:172.21.15.107:3300/0 conn(0x7f1918162020 0x7f1918162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.875+0000 7f1917e00640 1 -- 172.21.15.49:0/4149082255 >> v1:172.21.15.107:6789/0 conn(0x7f19180cf110 legacy=0x7f19180d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.875+0000 7f1917e00640 1 -- 172.21.15.49:0/4149082255 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19180cfad0 con 0x7f19180d1c90 2024-04-15T15:20:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.876+0000 7f1917e00640 1 --2- 172.21.15.49:0/4149082255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19180d1c90 0x7f19180d3390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f191808cd90 tx=0x7f191002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8703556a3bae44ac server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.876+0000 7f1916000640 1 -- 172.21.15.49:0/4149082255 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910034070 con 0x7f19180d1c90 2024-04-15T15:20:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.876+0000 7f1916000640 1 -- 172.21.15.49:0/4149082255 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1910030860 con 0x7f19180d1c90 2024-04-15T15:20:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.876+0000 7f1916000640 1 -- 172.21.15.49:0/4149082255 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910030c30 con 0x7f19180d1c90 2024-04-15T15:20:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.877+0000 7f191d200640 1 -- 172.21.15.49:0/4149082255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19180d1c90 msgr2=0x7f19180d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.877+0000 7f191d200640 1 --2- 172.21.15.49:0/4149082255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19180d1c90 0x7f19180d3390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f191808cd90 tx=0x7f191002b9a0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.877+0000 7f191d200640 1 -- 172.21.15.49:0/4149082255 shutdown_connections 2024-04-15T15:20:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.877+0000 7f191d200640 1 --2- 172.21.15.49:0/4149082255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19180d1c90 0x7f19180d3390 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.877+0000 7f191d200640 1 --2- 172.21.15.49:0/4149082255 >> v2:172.21.15.107:3300/0 conn(0x7f1918162020 0x7f1918162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.877+0000 7f191d200640 1 -- 172.21.15.49:0/4149082255 >> 172.21.15.49:0/4149082255 conn(0x7f191806cf00 msgr2=0x7f191806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.877+0000 7f191d200640 1 -- 172.21.15.49:0/4149082255 shutdown_connections 2024-04-15T15:20:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.877+0000 7f191d200640 1 -- 172.21.15.49:0/4149082255 wait complete. 2024-04-15T15:20:03.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.879+0000 7f191d200640 1 Processor -- start 2024-04-15T15:20:03.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.879+0000 7f191d200640 1 -- start start 2024-04-15T15:20:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.880+0000 7f191d200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19180cf110 0x7f191814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.880+0000 7f191d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19180d1c90 0x7f19180c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.880+0000 7f191d200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1918162020 0x7f19180ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.880+0000 7f191d200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f19180d2b90 con 0x7f1918162020 2024-04-15T15:20:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.880+0000 7f191d200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19180cf880 con 0x7f19180d1c90 2024-04-15T15:20:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.880+0000 7f191d200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f19180d2e70 con 0x7f19180cf110 2024-04-15T15:20:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.881+0000 7f1916a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19180d1c90 0x7f19180c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.881+0000 7f1917400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19180cf110 0x7f191814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.881+0000 7f1916a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19180d1c90 0x7f19180c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:33844/0 (socket says 172.21.15.49:33844) 2024-04-15T15:20:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.881+0000 7f1916a00640 1 -- 172.21.15.49:0/1795627141 learned_addr learned my addr 172.21.15.49:0/1795627141 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.881+0000 7f1917e00640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1918162020 0x7f19180ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.882+0000 7f1917e00640 1 -- 172.21.15.49:0/1795627141 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19180cf110 msgr2=0x7f191814ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.882+0000 7f1917e00640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19180cf110 0x7f191814ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.882+0000 7f1917e00640 1 -- 172.21.15.49:0/1795627141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19180d1c90 msgr2=0x7f19180c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.882+0000 7f1917e00640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19180d1c90 0x7f19180c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.882+0000 7f1917e00640 1 -- 172.21.15.49:0/1795627141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191814b160 con 0x7f1918162020 2024-04-15T15:20:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.882+0000 7f1917e00640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1918162020 0x7f19180ccd80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1910033000 tx=0x7f1910004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.882+0000 7f190f400640 1 -- 172.21.15.49:0/1795627141 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910048070 con 0x7f1918162020 2024-04-15T15:20:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.882+0000 7f190f400640 1 -- 172.21.15.49:0/1795627141 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1910030660 con 0x7f1918162020 2024-04-15T15:20:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.882+0000 7f191d200640 1 -- 172.21.15.49:0/1795627141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19180c8cc0 con 0x7f1918162020 2024-04-15T15:20:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.882+0000 7f190f400640 1 -- 172.21.15.49:0/1795627141 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910034040 con 0x7f1918162020 2024-04-15T15:20:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.883+0000 7f191d200640 1 -- 172.21.15.49:0/1795627141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19180c9280 con 0x7f1918162020 2024-04-15T15:20:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.884+0000 7f191d200640 1 -- 172.21.15.49:0/1795627141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19180755c0 con 0x7f1918162020 2024-04-15T15:20:03.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.885+0000 7f190f400640 1 -- 172.21.15.49:0/1795627141 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f191004b070 con 0x7f1918162020 2024-04-15T15:20:03.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.886+0000 7f190f400640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f18f806c8f0 0x7f18f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:03.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.886+0000 7f190f400640 1 -- 172.21.15.49:0/1795627141 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f19100ada40 con 0x7f1918162020 2024-04-15T15:20:03.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.886+0000 7f1917400640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f18f806c8f0 0x7f18f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:03.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.887+0000 7f1917400640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f18f806c8f0 0x7f18f806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f19080113e0 tx=0x7f190800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:03.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:03.890+0000 7f190f400640 1 -- 172.21.15.49:0/1795627141 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f191007b780 con 0x7f1918162020 2024-04-15T15:20:04.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.135+0000 7f191d200640 1 -- 172.21.15.49:0/1795627141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f191815f2c0 con 0x7f1918162020 2024-04-15T15:20:04.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.136+0000 7f190f400640 1 -- 172.21.15.49:0/1795627141 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f191007fa80 con 0x7f1918162020 2024-04-15T15:20:04.138 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:04.138 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:00.037472+0000","modified":"2024-04-15T15:20:03.151587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5067":{"gid":5067,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/1023024717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1023024717},{"type":"v1","addr":"172.21.15.49:6837","nonce":1023024717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5067,"qdb_cluster":[5067]},"id":2}]} 2024-04-15T15:20:04.139 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-04-15T15:20:04.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.139+0000 7f191d200640 1 -- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f18f806c8f0 msgr2=0x7f18f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.139+0000 7f191d200640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f18f806c8f0 0x7f18f806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f19080113e0 tx=0x7f190800b040 comp rx=0 tx=0).stop 2024-04-15T15:20:04.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.139+0000 7f191d200640 1 -- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1918162020 msgr2=0x7f19180ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.139+0000 7f191d200640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1918162020 0x7f19180ccd80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1910033000 tx=0x7f1910004730 comp rx=0 tx=0).stop 2024-04-15T15:20:04.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.139+0000 7f191d200640 1 -- 172.21.15.49:0/1795627141 shutdown_connections 2024-04-15T15:20:04.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.139+0000 7f191d200640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f18f806c8f0 0x7f18f806edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.139+0000 7f191d200640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1918162020 0x7f19180ccd80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.139+0000 7f191d200640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19180d1c90 0x7f19180c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.139+0000 7f191d200640 1 --2- 172.21.15.49:0/1795627141 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19180cf110 0x7f191814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.139+0000 7f191d200640 1 -- 172.21.15.49:0/1795627141 >> 172.21.15.49:0/1795627141 conn(0x7f191806cf00 msgr2=0x7f1918162c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:04.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.140+0000 7f191d200640 1 -- 172.21.15.49:0/1795627141 shutdown_connections 2024-04-15T15:20:04.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.140+0000 7f191d200640 1 -- 172.21.15.49:0/1795627141 wait complete. 2024-04-15T15:20:04.160 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:04.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.386+0000 7fb538a00640 1 Processor -- start 2024-04-15T15:20:04.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.387+0000 7fb538a00640 1 -- start start 2024-04-15T15:20:04.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.388+0000 7fb538a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5340b3c70 0x7fb5340d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.388+0000 7fb538a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb5340d1af0 0x7fb5340d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.388+0000 7fb538a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb53415ef90 con 0x7fb53415ebb0 2024-04-15T15:20:04.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.388+0000 7fb538a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fb5340cf090 con 0x7fb5340d1af0 2024-04-15T15:20:04.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.388+0000 7fb538a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb5340cf370 con 0x7fb5340b3c70 2024-04-15T15:20:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.388+0000 7fb532800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb53415ebb0 0x7fb534163060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:40372/0 (socket says 172.21.15.49:40372) 2024-04-15T15:20:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.388+0000 7fb532800640 1 -- 172.21.15.49:0/529981876 learned_addr learned my addr 172.21.15.49:0/529981876 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb533200640 1 --2- 172.21.15.49:0/529981876 >> v2:172.21.15.107:3300/0 conn(0x7fb5340d1af0 0x7fb5340d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb52be00640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5340b3c70 0x7fb5340d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747560228 0 0) 0x7fb53415ef90 con 0x7fb53415ebb0 2024-04-15T15:20:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb518003520 con 0x7fb53415ebb0 2024-04-15T15:20:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1689622595 0 0) 0x7fb518003520 con 0x7fb53415ebb0 2024-04-15T15:20:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5340b3c70 msgr2=0x7fb5340d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5340b3c70 0x7fb5340d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 >> v2:172.21.15.107:3300/0 conn(0x7fb5340d1af0 msgr2=0x7fb5340d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 --2- 172.21.15.49:0/529981876 >> v2:172.21.15.107:3300/0 conn(0x7fb5340d1af0 0x7fb5340d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5340cf650 con 0x7fb53415ebb0 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fb524001670 con 0x7fb53415ebb0 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 >> v1:172.21.15.49:6789/0 conn(0x7fb53415ebb0 legacy=0x7fb534163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.389+0000 7fb531e00640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5180039b0 0x7fb518003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb531e00640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb518004800 0x7fb518004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb531e00640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb518005180 0x7fb51800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5340cf090 con 0x7fb5180039b0 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5340cf370 con 0x7fb518005180 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb518004630 con 0x7fb518004800 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb532800640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5180039b0 0x7fb518003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb533200640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb518005180 0x7fb51800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb52be00640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb518004800 0x7fb518004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb532800640 1 -- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb518004800 msgr2=0x7fb518004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb532800640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb518004800 0x7fb518004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb532800640 1 -- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb518005180 msgr2=0x7fb51800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb532800640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb518005180 0x7fb51800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb532800640 1 -- 172.21.15.49:0/529981876 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb518003520 con 0x7fb5180039b0 2024-04-15T15:20:04.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.390+0000 7fb533200640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb518005180 0x7fb51800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:04.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.391+0000 7fb532800640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5180039b0 0x7fb518003dd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb52400c290 tx=0x7fb518005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec3ea40e583caa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.392+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524007d50 con 0x7fb5180039b0 2024-04-15T15:20:04.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.392+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb524030070 con 0x7fb5180039b0 2024-04-15T15:20:04.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.392+0000 7fb531e00640 1 -- 172.21.15.49:0/529981876 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52402cd80 con 0x7fb5180039b0 2024-04-15T15:20:04.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.392+0000 7fb538a00640 1 -- 172.21.15.49:0/529981876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5180039b0 msgr2=0x7fb518003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.392+0000 7fb538a00640 1 --2- 172.21.15.49:0/529981876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5180039b0 0x7fb518003dd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb52400c290 tx=0x7fb518005830 comp rx=0 tx=0).stop 2024-04-15T15:20:04.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.393+0000 7fb533200640 1 -- 172.21.15.49:0/529981876 reap_dead start 2024-04-15T15:20:04.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.393+0000 7fb538a00640 1 -- 172.21.15.49:0/529981876 shutdown_connections 2024-04-15T15:20:04.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.393+0000 7fb538a00640 1 -- 172.21.15.49:0/529981876 >> 172.21.15.49:0/529981876 conn(0x7fb53406d040 msgr2=0x7fb53406d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:04.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.393+0000 7fb538a00640 1 -- 172.21.15.49:0/529981876 shutdown_connections 2024-04-15T15:20:04.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.393+0000 7fb538a00640 1 -- 172.21.15.49:0/529981876 wait complete. 2024-04-15T15:20:04.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.395+0000 7fb538a00640 1 Processor -- start 2024-04-15T15:20:04.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.395+0000 7fb538a00640 1 -- start start 2024-04-15T15:20:04.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb538a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5180039b0 0x7fb53415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb538a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5340ccef0 0x7fb5340cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb538a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5341479f0 0x7fb534147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb538a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5340cfef0 con 0x7fb5341479f0 2024-04-15T15:20:04.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb538a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5340cfc10 con 0x7fb5340ccef0 2024-04-15T15:20:04.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb538a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb5340d15b0 con 0x7fb5180039b0 2024-04-15T15:20:04.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb533200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5341479f0 0x7fb534147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb533200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5341479f0 0x7fb534147e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42822/0 (socket says 172.21.15.49:42822) 2024-04-15T15:20:04.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb533200640 1 -- 172.21.15.49:0/3423161183 learned_addr learned my addr 172.21.15.49:0/3423161183 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:04.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb52be00640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5340ccef0 0x7fb5340cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.396+0000 7fb532800640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5180039b0 0x7fb53415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb533200640 1 -- 172.21.15.49:0/3423161183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5180039b0 msgr2=0x7fb53415b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb533200640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5180039b0 0x7fb53415b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb533200640 1 -- 172.21.15.49:0/3423161183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5340ccef0 msgr2=0x7fb5340cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb533200640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5340ccef0 0x7fb5340cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb533200640 1 -- 172.21.15.49:0/3423161183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb534146950 con 0x7fb5341479f0 2024-04-15T15:20:04.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb532800640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5180039b0 0x7fb53415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:04.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb533200640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5341479f0 0x7fb534147e70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009fd0 tx=0x7fb52c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb52aa00640 1 -- 172.21.15.49:0/3423161183 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c016620 con 0x7fb5341479f0 2024-04-15T15:20:04.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb52aa00640 1 -- 172.21.15.49:0/3423161183 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52c01b070 con 0x7fb5341479f0 2024-04-15T15:20:04.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb538a00640 1 -- 172.21.15.49:0/3423161183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb534146c30 con 0x7fb5341479f0 2024-04-15T15:20:04.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb52aa00640 1 -- 172.21.15.49:0/3423161183 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c0169e0 con 0x7fb5341479f0 2024-04-15T15:20:04.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.397+0000 7fb538a00640 1 -- 172.21.15.49:0/3423161183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5341471f0 con 0x7fb5341479f0 2024-04-15T15:20:04.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.399+0000 7fb538a00640 1 -- 172.21.15.49:0/3423161183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5340755c0 con 0x7fb5341479f0 2024-04-15T15:20:04.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.400+0000 7fb52aa00640 1 -- 172.21.15.49:0/3423161183 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb52c005d10 con 0x7fb5341479f0 2024-04-15T15:20:04.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.400+0000 7fb52aa00640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb51006c920 0x7fb51006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.401+0000 7fb52aa00640 1 -- 172.21.15.49:0/3423161183 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb52c08c6a0 con 0x7fb5341479f0 2024-04-15T15:20:04.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.401+0000 7fb532800640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb51006c920 0x7fb51006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.402+0000 7fb532800640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb51006c920 0x7fb51006ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb524034000 tx=0x7fb524007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.405+0000 7fb52aa00640 1 -- 172.21.15.49:0/3423161183 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb52c025020 con 0x7fb5341479f0 2024-04-15T15:20:04.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.610+0000 7fb538a00640 1 -- 172.21.15.49:0/3423161183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5340d2540 con 0x7fb5341479f0 2024-04-15T15:20:04.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.613+0000 7fb52aa00640 1 -- 172.21.15.49:0/3423161183 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16669 (secure 0 0 0) 0x7fb52c05aa30 con 0x7fb5341479f0 2024-04-15T15:20:04.615 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:04.616 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:01.591001+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:04.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 -- 172.21.15.49:0/3423161183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb51006c920 msgr2=0x7fb51006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb51006c920 0x7fb51006ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb524034000 tx=0x7fb524007000 comp rx=0 tx=0).stop 2024-04-15T15:20:04.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 -- 172.21.15.49:0/3423161183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5341479f0 msgr2=0x7fb534147e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5341479f0 0x7fb534147e70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009fd0 tx=0x7fb52c00f520 comp rx=0 tx=0).stop 2024-04-15T15:20:04.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 -- 172.21.15.49:0/3423161183 shutdown_connections 2024-04-15T15:20:04.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb51006c920 0x7fb51006ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5341479f0 0x7fb534147e70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5340ccef0 0x7fb5340cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 --2- 172.21.15.49:0/3423161183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5180039b0 0x7fb53415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 -- 172.21.15.49:0/3423161183 >> 172.21.15.49:0/3423161183 conn(0x7fb53406d040 msgr2=0x7fb5340d2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:04.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 -- 172.21.15.49:0/3423161183 shutdown_connections 2024-04-15T15:20:04.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.616+0000 7fb538a00640 1 -- 172.21.15.49:0/3423161183 wait complete. 2024-04-15T15:20:04.640 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-15T15:20:04.640 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:20:04.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.855+0000 7f7a94200640 1 Processor -- start 2024-04-15T15:20:04.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.855+0000 7f7a94200640 1 -- start start 2024-04-15T15:20:04.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.857+0000 7f7a94200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7a8c0728f0 0x7f7a8c15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.857+0000 7f7a94200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c0affc0 0x7f7a8c1601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.857+0000 7f7a94200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8c1485d0 con 0x7f7a8c1481f0 2024-04-15T15:20:04.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.857+0000 7f7a94200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f7a8c15ffc0 con 0x7f7a8c0728f0 2024-04-15T15:20:04.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.857+0000 7f7a94200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c0aef20 con 0x7f7a8c0affc0 2024-04-15T15:20:04.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.858+0000 7f7a92200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c0affc0 0x7f7a8c1601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.858+0000 7f7a90e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7a8c0728f0 0x7f7a8c15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.858+0000 7f7a92200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c0affc0 0x7f7a8c1601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:45738/0 (socket says 172.21.15.49:45738) 2024-04-15T15:20:04.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.858+0000 7f7a92200640 1 -- 172.21.15.49:0/2290848373 learned_addr learned my addr 172.21.15.49:0/2290848373 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:04.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.859+0000 7f7a90e00640 1 -- 172.21.15.49:0/2290848373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c0affc0 msgr2=0x7f7a8c1601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:04.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.859+0000 7f7a90e00640 1 --2- 172.21.15.49:0/2290848373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c0affc0 0x7f7a8c1601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.859+0000 7f7a90e00640 1 -- 172.21.15.49:0/2290848373 >> v1:172.21.15.107:6789/0 conn(0x7f7a8c1481f0 legacy=0x7f7a8c0721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.859+0000 7f7a90e00640 1 -- 172.21.15.49:0/2290848373 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c0af200 con 0x7f7a8c0728f0 2024-04-15T15:20:04.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.859+0000 7f7a92200640 1 --2- 172.21.15.49:0/2290848373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c0affc0 0x7f7a8c1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.860+0000 7f7a90e00640 1 --2- 172.21.15.49:0/2290848373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c0728f0 0x7f7a8c15fa80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0097a0 tx=0x7f7a7c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b369011a980f3e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.860+0000 7f7a8be00640 1 -- 172.21.15.49:0/2290848373 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c035020 con 0x7f7a8c0728f0 2024-04-15T15:20:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.860+0000 7f7a8be00640 1 -- 172.21.15.49:0/2290848373 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a7c031810 con 0x7f7a8c0728f0 2024-04-15T15:20:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.860+0000 7f7a8be00640 1 -- 172.21.15.49:0/2290848373 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c031be0 con 0x7f7a8c0728f0 2024-04-15T15:20:04.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.861+0000 7f7a94200640 1 -- 172.21.15.49:0/2290848373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c0728f0 msgr2=0x7f7a8c15fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.861+0000 7f7a94200640 1 --2- 172.21.15.49:0/2290848373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c0728f0 0x7f7a8c15fa80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0097a0 tx=0x7f7a7c02b8b0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.861+0000 7f7a94200640 1 -- 172.21.15.49:0/2290848373 shutdown_connections 2024-04-15T15:20:04.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.861+0000 7f7a94200640 1 --2- 172.21.15.49:0/2290848373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c0affc0 0x7f7a8c1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.861+0000 7f7a94200640 1 --2- 172.21.15.49:0/2290848373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c0728f0 0x7f7a8c15fa80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.861+0000 7f7a94200640 1 -- 172.21.15.49:0/2290848373 >> 172.21.15.49:0/2290848373 conn(0x7f7a8c0b27b0 msgr2=0x7f7a8c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:04.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.861+0000 7f7a94200640 1 -- 172.21.15.49:0/2290848373 shutdown_connections 2024-04-15T15:20:04.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.861+0000 7f7a94200640 1 -- 172.21.15.49:0/2290848373 wait complete. 2024-04-15T15:20:04.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.863+0000 7f7a94200640 1 Processor -- start 2024-04-15T15:20:04.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.863+0000 7f7a94200640 1 -- start start 2024-04-15T15:20:04.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.864+0000 7f7a94200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c0728f0 0x7f7a8c1f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.864+0000 7f7a94200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a8c0affc0 0x7f7a8c140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.864+0000 7f7a94200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c1481f0 0x7f7a8c06fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.864+0000 7f7a94200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0afaa0 con 0x7f7a8c0affc0 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.864+0000 7f7a94200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0af7c0 con 0x7f7a8c0728f0 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.864+0000 7f7a94200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c0b5560 con 0x7f7a8c1481f0 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.864+0000 7f7a90e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a8c0affc0 0x7f7a8c140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.865+0000 7f7a90e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a8c0affc0 0x7f7a8c140fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42836/0 (socket says 172.21.15.49:42836) 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.865+0000 7f7a90e00640 1 -- 172.21.15.49:0/3535945276 learned_addr learned my addr 172.21.15.49:0/3535945276 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.865+0000 7f7a91800640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c0728f0 0x7f7a8c1f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.865+0000 7f7a92200640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c1481f0 0x7f7a8c06fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.865+0000 7f7a90e00640 1 -- 172.21.15.49:0/3535945276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c1481f0 msgr2=0x7f7a8c06fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.865+0000 7f7a90e00640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c1481f0 0x7f7a8c06fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.865+0000 7f7a90e00640 1 -- 172.21.15.49:0/3535945276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c0728f0 msgr2=0x7f7a8c1f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.865+0000 7f7a90e00640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c0728f0 0x7f7a8c1f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:04.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.865+0000 7f7a90e00640 1 -- 172.21.15.49:0/3535945276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1f85a0 con 0x7f7a8c0affc0 2024-04-15T15:20:04.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.865+0000 7f7a90e00640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a8c0affc0 0x7f7a8c140fb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c02bec0 tx=0x7f7a7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.866+0000 7f7a89600640 1 -- 172.21.15.49:0/3535945276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c041070 con 0x7f7a8c0affc0 2024-04-15T15:20:04.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.866+0000 7f7a89600640 1 -- 172.21.15.49:0/3535945276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a7c03ad00 con 0x7f7a8c0affc0 2024-04-15T15:20:04.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.866+0000 7f7a89600640 1 -- 172.21.15.49:0/3535945276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c035040 con 0x7f7a8c0affc0 2024-04-15T15:20:04.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.866+0000 7f7a94200640 1 -- 172.21.15.49:0/3535945276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1f8880 con 0x7f7a8c0affc0 2024-04-15T15:20:04.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.866+0000 7f7a94200640 1 -- 172.21.15.49:0/3535945276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1f8e40 con 0x7f7a8c0affc0 2024-04-15T15:20:04.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.867+0000 7f7a94200640 1 -- 172.21.15.49:0/3535945276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c068c80 con 0x7f7a8c0affc0 2024-04-15T15:20:04.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.868+0000 7f7a89600640 1 -- 172.21.15.49:0/3535945276 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a7c04c060 con 0x7f7a8c0affc0 2024-04-15T15:20:04.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.869+0000 7f7a89600640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7a7006c970 0x7f7a7006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:04.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.869+0000 7f7a89600640 1 -- 172.21.15.49:0/3535945276 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f7a7c0ad510 con 0x7f7a8c0affc0 2024-04-15T15:20:04.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.869+0000 7f7a91800640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7a7006c970 0x7f7a7006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:04.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.870+0000 7f7a91800640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7a7006c970 0x7f7a7006ee30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a80000f60 tx=0x7f7a80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:04.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:04.873+0000 7f7a89600640 1 -- 172.21.15.49:0/3535945276 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a7c07bab0 con 0x7f7a8c0affc0 2024-04-15T15:20:05.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.075+0000 7f7a94200640 1 -- 172.21.15.49:0/3535945276 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a8c15eb40 con 0x7f7a7006c970 2024-04-15T15:20:05.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.089+0000 7f7a89600640 1 -- 172.21.15.49:0/3535945276 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7f7a8c15eb40 con 0x7f7a7006c970 2024-04-15T15:20:05.094 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:05.099 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 -- 172.21.15.49:0/3535945276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7a7006c970 msgr2=0x7f7a7006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7a7006c970 0x7f7a7006ee30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a80000f60 tx=0x7f7a80009040 comp rx=0 tx=0).stop 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 -- 172.21.15.49:0/3535945276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a8c0affc0 msgr2=0x7f7a8c140fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a8c0affc0 0x7f7a8c140fb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c02bec0 tx=0x7f7a7c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 -- 172.21.15.49:0/3535945276 shutdown_connections 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7a7006c970 0x7f7a7006ee30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a8c1481f0 0x7f7a8c06fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a8c0affc0 0x7f7a8c140fb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 --2- 172.21.15.49:0/3535945276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c0728f0 0x7f7a8c1f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 -- 172.21.15.49:0/3535945276 >> 172.21.15.49:0/3535945276 conn(0x7f7a8c0b27b0 msgr2=0x7f7a8c145c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 -- 172.21.15.49:0/3535945276 shutdown_connections 2024-04-15T15:20:05.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.094+0000 7f7a94200640 1 -- 172.21.15.49:0/3535945276 wait complete. 2024-04-15T15:20:05.117 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-15T15:20:04.796908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237104,"kb_used_data":4272,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737616,"statfs":{"total":773094113280,"available":772851318784,"internally_reserved":0,"allocated":4374528,"data_stored":2409612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002345"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610391+0000","last_change":"2024-04-15T15:20:00.609222+0000","last_active":"2024-04-15T15:20:01.610391+0000","last_peered":"2024-04-15T15:20:01.610391+0000","last_clean":"2024-04-15T15:20:01.610391+0000","last_became_active":"2024-04-15T15:20:00.608895+0000","last_became_peered":"2024-04-15T15:20:00.608895+0000","last_unstale":"2024-04-15T15:20:01.610391+0000","last_undegraded":"2024-04-15T15:20:01.610391+0000","last_fullsized":"2024-04-15T15:20:01.610391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:51:04.022267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906871+0000","last_change":"2024-04-15T15:20:00.622797+0000","last_active":"2024-04-15T15:20:01.906871+0000","last_peered":"2024-04-15T15:20:01.906871+0000","last_clean":"2024-04-15T15:20:01.906871+0000","last_became_active":"2024-04-15T15:20:00.621060+0000","last_became_peered":"2024-04-15T15:20:00.621060+0000","last_unstale":"2024-04-15T15:20:01.906871+0000","last_undegraded":"2024-04-15T15:20:01.906871+0000","last_fullsized":"2024-04-15T15:20:01.906871+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:54.362725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608515+0000","last_change":"2024-04-15T15:20:00.612626+0000","last_active":"2024-04-15T15:20:01.608515+0000","last_peered":"2024-04-15T15:20:01.608515+0000","last_clean":"2024-04-15T15:20:01.608515+0000","last_became_active":"2024-04-15T15:20:00.611057+0000","last_became_peered":"2024-04-15T15:20:00.611057+0000","last_unstale":"2024-04-15T15:20:01.608515+0000","last_undegraded":"2024-04-15T15:20:01.608515+0000","last_fullsized":"2024-04-15T15:20:01.608515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:17:36.203246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624346+0000","last_change":"2024-04-15T15:20:00.621018+0000","last_active":"2024-04-15T15:20:01.624346+0000","last_peered":"2024-04-15T15:20:01.624346+0000","last_clean":"2024-04-15T15:20:01.624346+0000","last_became_active":"2024-04-15T15:20:00.620101+0000","last_became_peered":"2024-04-15T15:20:00.620101+0000","last_unstale":"2024-04-15T15:20:01.624346+0000","last_undegraded":"2024-04-15T15:20:01.624346+0000","last_fullsized":"2024-04-15T15:20:01.624346+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:00:21.659268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903567+0000","last_change":"2024-04-15T15:20:00.622082+0000","last_active":"2024-04-15T15:20:01.903567+0000","last_peered":"2024-04-15T15:20:01.903567+0000","last_clean":"2024-04-15T15:20:01.903567+0000","last_became_active":"2024-04-15T15:20:00.621734+0000","last_became_peered":"2024-04-15T15:20:00.621734+0000","last_unstale":"2024-04-15T15:20:01.903567+0000","last_undegraded":"2024-04-15T15:20:01.903567+0000","last_fullsized":"2024-04-15T15:20:01.903567+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:26:22.792791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905334+0000","last_change":"2024-04-15T15:20:00.612629+0000","last_active":"2024-04-15T15:20:01.905334+0000","last_peered":"2024-04-15T15:20:01.905334+0000","last_clean":"2024-04-15T15:20:01.905334+0000","last_became_active":"2024-04-15T15:20:00.609581+0000","last_became_peered":"2024-04-15T15:20:00.609581+0000","last_unstale":"2024-04-15T15:20:01.905334+0000","last_undegraded":"2024-04-15T15:20:01.905334+0000","last_fullsized":"2024-04-15T15:20:01.905334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:02:52.890702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.605290+0000","last_change":"2024-04-15T15:20:00.609734+0000","last_active":"2024-04-15T15:20:01.605290+0000","last_peered":"2024-04-15T15:20:01.605290+0000","last_clean":"2024-04-15T15:20:01.605290+0000","last_became_active":"2024-04-15T15:20:00.609154+0000","last_became_peered":"2024-04-15T15:20:00.609154+0000","last_unstale":"2024-04-15T15:20:01.605290+0000","last_undegraded":"2024-04-15T15:20:01.605290+0000","last_fullsized":"2024-04-15T15:20:01.605290+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:01:26.510445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.604878+0000","last_change":"2024-04-15T15:20:00.619025+0000","last_active":"2024-04-15T15:20:01.604878+0000","last_peered":"2024-04-15T15:20:01.604878+0000","last_clean":"2024-04-15T15:20:01.604878+0000","last_became_active":"2024-04-15T15:20:00.618572+0000","last_became_peered":"2024-04-15T15:20:00.618572+0000","last_unstale":"2024-04-15T15:20:01.604878+0000","last_undegraded":"2024-04-15T15:20:01.604878+0000","last_fullsized":"2024-04-15T15:20:01.604878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:04:28.710555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905001+0000","last_change":"2024-04-15T15:20:00.617554+0000","last_active":"2024-04-15T15:20:01.905001+0000","last_peered":"2024-04-15T15:20:01.905001+0000","last_clean":"2024-04-15T15:20:01.905001+0000","last_became_active":"2024-04-15T15:20:00.617000+0000","last_became_peered":"2024-04-15T15:20:00.617000+0000","last_unstale":"2024-04-15T15:20:01.905001+0000","last_undegraded":"2024-04-15T15:20:01.905001+0000","last_fullsized":"2024-04-15T15:20:01.905001+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:04:16.834409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624693+0000","last_change":"2024-04-15T15:20:00.624239+0000","last_active":"2024-04-15T15:20:01.624693+0000","last_peered":"2024-04-15T15:20:01.624693+0000","last_clean":"2024-04-15T15:20:01.624693+0000","last_became_active":"2024-04-15T15:20:00.623613+0000","last_became_peered":"2024-04-15T15:20:00.623613+0000","last_unstale":"2024-04-15T15:20:01.624693+0000","last_undegraded":"2024-04-15T15:20:01.624693+0000","last_fullsized":"2024-04-15T15:20:01.624693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:38:12.140232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.607602+0000","last_change":"2024-04-15T15:20:00.610848+0000","last_active":"2024-04-15T15:20:01.607602+0000","last_peered":"2024-04-15T15:20:01.607602+0000","last_clean":"2024-04-15T15:20:01.607602+0000","last_became_active":"2024-04-15T15:20:00.610572+0000","last_became_peered":"2024-04-15T15:20:00.610572+0000","last_unstale":"2024-04-15T15:20:01.607602+0000","last_undegraded":"2024-04-15T15:20:01.607602+0000","last_fullsized":"2024-04-15T15:20:01.607602+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:36:47.757839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608493+0000","last_change":"2024-04-15T15:20:00.610374+0000","last_active":"2024-04-15T15:20:01.608493+0000","last_peered":"2024-04-15T15:20:01.608493+0000","last_clean":"2024-04-15T15:20:01.608493+0000","last_became_active":"2024-04-15T15:20:00.609999+0000","last_became_peered":"2024-04-15T15:20:00.609999+0000","last_unstale":"2024-04-15T15:20:01.608493+0000","last_undegraded":"2024-04-15T15:20:01.608493+0000","last_fullsized":"2024-04-15T15:20:01.608493+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:35:41.539419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903431+0000","last_change":"2024-04-15T15:20:00.617475+0000","last_active":"2024-04-15T15:20:01.903431+0000","last_peered":"2024-04-15T15:20:01.903431+0000","last_clean":"2024-04-15T15:20:01.903431+0000","last_became_active":"2024-04-15T15:20:00.615756+0000","last_became_peered":"2024-04-15T15:20:00.615756+0000","last_unstale":"2024-04-15T15:20:01.903431+0000","last_undegraded":"2024-04-15T15:20:01.903431+0000","last_fullsized":"2024-04-15T15:20:01.903431+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:52:57.430907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905447+0000","last_change":"2024-04-15T15:20:00.628890+0000","last_active":"2024-04-15T15:20:01.905447+0000","last_peered":"2024-04-15T15:20:01.905447+0000","last_clean":"2024-04-15T15:20:01.905447+0000","last_became_active":"2024-04-15T15:20:00.628579+0000","last_became_peered":"2024-04-15T15:20:00.628579+0000","last_unstale":"2024-04-15T15:20:01.905447+0000","last_undegraded":"2024-04-15T15:20:01.905447+0000","last_fullsized":"2024-04-15T15:20:01.905447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:00:29.913810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608265+0000","last_change":"2024-04-15T15:20:00.609953+0000","last_active":"2024-04-15T15:20:01.608265+0000","last_peered":"2024-04-15T15:20:01.608265+0000","last_clean":"2024-04-15T15:20:01.608265+0000","last_became_active":"2024-04-15T15:20:00.609645+0000","last_became_peered":"2024-04-15T15:20:00.609645+0000","last_unstale":"2024-04-15T15:20:01.608265+0000","last_undegraded":"2024-04-15T15:20:01.608265+0000","last_fullsized":"2024-04-15T15:20:01.608265+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:30:16.127825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.606218+0000","last_change":"2024-04-15T15:20:00.616605+0000","last_active":"2024-04-15T15:20:01.606218+0000","last_peered":"2024-04-15T15:20:01.606218+0000","last_clean":"2024-04-15T15:20:01.606218+0000","last_became_active":"2024-04-15T15:20:00.616258+0000","last_became_peered":"2024-04-15T15:20:00.616258+0000","last_unstale":"2024-04-15T15:20:01.606218+0000","last_undegraded":"2024-04-15T15:20:01.606218+0000","last_fullsized":"2024-04-15T15:20:01.606218+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:17:36.203246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.606119+0000","last_change":"2024-04-15T15:20:00.615339+0000","last_active":"2024-04-15T15:20:01.606119+0000","last_peered":"2024-04-15T15:20:01.606119+0000","last_clean":"2024-04-15T15:20:01.606119+0000","last_became_active":"2024-04-15T15:20:00.613420+0000","last_became_peered":"2024-04-15T15:20:00.613420+0000","last_unstale":"2024-04-15T15:20:01.606119+0000","last_undegraded":"2024-04-15T15:20:01.606119+0000","last_fullsized":"2024-04-15T15:20:01.606119+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:35:41.539419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906912+0000","last_change":"2024-04-15T15:20:00.623316+0000","last_active":"2024-04-15T15:20:01.906912+0000","last_peered":"2024-04-15T15:20:01.906912+0000","last_clean":"2024-04-15T15:20:01.906912+0000","last_became_active":"2024-04-15T15:20:00.623048+0000","last_became_peered":"2024-04-15T15:20:00.623048+0000","last_unstale":"2024-04-15T15:20:01.906912+0000","last_undegraded":"2024-04-15T15:20:01.906912+0000","last_fullsized":"2024-04-15T15:20:01.906912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:27.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906162+0000","last_change":"2024-04-15T15:20:00.615947+0000","last_active":"2024-04-15T15:20:01.906162+0000","last_peered":"2024-04-15T15:20:01.906162+0000","last_clean":"2024-04-15T15:20:01.906162+0000","last_became_active":"2024-04-15T15:20:00.615629+0000","last_became_peered":"2024-04-15T15:20:00.615629+0000","last_unstale":"2024-04-15T15:20:01.906162+0000","last_undegraded":"2024-04-15T15:20:01.906162+0000","last_fullsized":"2024-04-15T15:20:01.906162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:53.187316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903746+0000","last_change":"2024-04-15T15:20:00.620795+0000","last_active":"2024-04-15T15:20:01.903746+0000","last_peered":"2024-04-15T15:20:01.903746+0000","last_clean":"2024-04-15T15:20:01.903746+0000","last_became_active":"2024-04-15T15:20:00.620452+0000","last_became_peered":"2024-04-15T15:20:00.620452+0000","last_unstale":"2024-04-15T15:20:01.903746+0000","last_undegraded":"2024-04-15T15:20:01.903746+0000","last_fullsized":"2024-04-15T15:20:01.903746+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:43:39.547287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904403+0000","last_change":"2024-04-15T15:20:00.609970+0000","last_active":"2024-04-15T15:20:01.904403+0000","last_peered":"2024-04-15T15:20:01.904403+0000","last_clean":"2024-04-15T15:20:01.904403+0000","last_became_active":"2024-04-15T15:20:00.609657+0000","last_became_peered":"2024-04-15T15:20:00.609657+0000","last_unstale":"2024-04-15T15:20:01.904403+0000","last_undegraded":"2024-04-15T15:20:01.904403+0000","last_fullsized":"2024-04-15T15:20:01.904403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:34:17.016954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904529+0000","last_change":"2024-04-15T15:20:00.617412+0000","last_active":"2024-04-15T15:20:01.904529+0000","last_peered":"2024-04-15T15:20:01.904529+0000","last_clean":"2024-04-15T15:20:01.904529+0000","last_became_active":"2024-04-15T15:20:00.617083+0000","last_became_peered":"2024-04-15T15:20:00.617083+0000","last_unstale":"2024-04-15T15:20:01.904529+0000","last_undegraded":"2024-04-15T15:20:01.904529+0000","last_fullsized":"2024-04-15T15:20:01.904529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:47:52.358011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620254+0000","last_change":"2024-04-15T15:20:00.601947+0000","last_active":"2024-04-15T15:20:01.620254+0000","last_peered":"2024-04-15T15:20:01.620254+0000","last_clean":"2024-04-15T15:20:01.620254+0000","last_became_active":"2024-04-15T15:20:00.601497+0000","last_became_peered":"2024-04-15T15:20:00.601497+0000","last_unstale":"2024-04-15T15:20:01.620254+0000","last_undegraded":"2024-04-15T15:20:01.620254+0000","last_fullsized":"2024-04-15T15:20:01.620254+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:51:04.022267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608185+0000","last_change":"2024-04-15T15:20:00.614918+0000","last_active":"2024-04-15T15:20:01.608185+0000","last_peered":"2024-04-15T15:20:01.608185+0000","last_clean":"2024-04-15T15:20:01.608185+0000","last_became_active":"2024-04-15T15:20:00.614630+0000","last_became_peered":"2024-04-15T15:20:00.614630+0000","last_unstale":"2024-04-15T15:20:01.608185+0000","last_undegraded":"2024-04-15T15:20:01.608185+0000","last_fullsized":"2024-04-15T15:20:01.608185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:38:12.140232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624909+0000","last_change":"2024-04-15T15:20:00.613167+0000","last_active":"2024-04-15T15:20:01.624909+0000","last_peered":"2024-04-15T15:20:01.624909+0000","last_clean":"2024-04-15T15:20:01.624909+0000","last_became_active":"2024-04-15T15:20:00.612760+0000","last_became_peered":"2024-04-15T15:20:00.612760+0000","last_unstale":"2024-04-15T15:20:01.624909+0000","last_undegraded":"2024-04-15T15:20:01.624909+0000","last_fullsized":"2024-04-15T15:20:01.624909+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:30:16.127825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.907193+0000","last_change":"2024-04-15T15:20:00.612735+0000","last_active":"2024-04-15T15:20:01.907193+0000","last_peered":"2024-04-15T15:20:01.907193+0000","last_clean":"2024-04-15T15:20:01.907193+0000","last_became_active":"2024-04-15T15:20:00.612258+0000","last_became_peered":"2024-04-15T15:20:00.612258+0000","last_unstale":"2024-04-15T15:20:01.907193+0000","last_undegraded":"2024-04-15T15:20:01.907193+0000","last_fullsized":"2024-04-15T15:20:01.907193+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:53:46.359850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624975+0000","last_change":"2024-04-15T15:20:00.620494+0000","last_active":"2024-04-15T15:20:01.624975+0000","last_peered":"2024-04-15T15:20:01.624975+0000","last_clean":"2024-04-15T15:20:01.624975+0000","last_became_active":"2024-04-15T15:20:00.620229+0000","last_became_peered":"2024-04-15T15:20:00.620229+0000","last_unstale":"2024-04-15T15:20:01.624975+0000","last_undegraded":"2024-04-15T15:20:01.624975+0000","last_fullsized":"2024-04-15T15:20:01.624975+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:35:41.539419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609012+0000","last_change":"2024-04-15T15:20:00.603794+0000","last_active":"2024-04-15T15:20:01.609012+0000","last_peered":"2024-04-15T15:20:01.609012+0000","last_clean":"2024-04-15T15:20:01.609012+0000","last_became_active":"2024-04-15T15:20:00.603460+0000","last_became_peered":"2024-04-15T15:20:00.603460+0000","last_unstale":"2024-04-15T15:20:01.609012+0000","last_undegraded":"2024-04-15T15:20:01.609012+0000","last_fullsized":"2024-04-15T15:20:01.609012+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:00:29.913810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904947+0000","last_change":"2024-04-15T15:20:00.618347+0000","last_active":"2024-04-15T15:20:01.904947+0000","last_peered":"2024-04-15T15:20:01.904947+0000","last_clean":"2024-04-15T15:20:01.904947+0000","last_became_active":"2024-04-15T15:20:00.618041+0000","last_became_peered":"2024-04-15T15:20:00.618041+0000","last_unstale":"2024-04-15T15:20:01.904947+0000","last_undegraded":"2024-04-15T15:20:01.904947+0000","last_fullsized":"2024-04-15T15:20:01.904947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:05:39.404242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.606011+0000","last_change":"2024-04-15T15:20:00.611489+0000","last_active":"2024-04-15T15:20:01.606011+0000","last_peered":"2024-04-15T15:20:01.606011+0000","last_clean":"2024-04-15T15:20:01.606011+0000","last_became_active":"2024-04-15T15:20:00.607617+0000","last_became_peered":"2024-04-15T15:20:00.607617+0000","last_unstale":"2024-04-15T15:20:01.606011+0000","last_undegraded":"2024-04-15T15:20:01.606011+0000","last_fullsized":"2024-04-15T15:20:01.606011+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:30:16.127825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609663+0000","last_change":"2024-04-15T15:20:00.616963+0000","last_active":"2024-04-15T15:20:01.609663+0000","last_peered":"2024-04-15T15:20:01.609663+0000","last_clean":"2024-04-15T15:20:01.609663+0000","last_became_active":"2024-04-15T15:20:00.615614+0000","last_became_peered":"2024-04-15T15:20:00.615614+0000","last_unstale":"2024-04-15T15:20:01.609663+0000","last_undegraded":"2024-04-15T15:20:01.609663+0000","last_fullsized":"2024-04-15T15:20:01.609663+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:28:57.704150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.606878+0000","last_change":"2024-04-15T15:20:00.607968+0000","last_active":"2024-04-15T15:20:01.606878+0000","last_peered":"2024-04-15T15:20:01.606878+0000","last_clean":"2024-04-15T15:20:01.606878+0000","last_became_active":"2024-04-15T15:20:00.606951+0000","last_became_peered":"2024-04-15T15:20:00.606951+0000","last_unstale":"2024-04-15T15:20:01.606878+0000","last_undegraded":"2024-04-15T15:20:01.606878+0000","last_fullsized":"2024-04-15T15:20:01.606878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:00:29.913810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620482+0000","last_change":"2024-04-15T15:20:00.607623+0000","last_active":"2024-04-15T15:20:01.620482+0000","last_peered":"2024-04-15T15:20:01.620482+0000","last_clean":"2024-04-15T15:20:01.620482+0000","last_became_active":"2024-04-15T15:20:00.606698+0000","last_became_peered":"2024-04-15T15:20:00.606698+0000","last_unstale":"2024-04-15T15:20:01.620482+0000","last_undegraded":"2024-04-15T15:20:01.620482+0000","last_fullsized":"2024-04-15T15:20:01.620482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:20:29.604777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610534+0000","last_change":"2024-04-15T15:19:59.562229+0000","last_active":"2024-04-15T15:20:01.610534+0000","last_peered":"2024-04-15T15:20:01.610534+0000","last_clean":"2024-04-15T15:20:01.610534+0000","last_became_active":"2024-04-15T15:19:59.561906+0000","last_became_peered":"2024-04-15T15:19:59.561906+0000","last_unstale":"2024-04-15T15:20:01.610534+0000","last_undegraded":"2024-04-15T15:20:01.610534+0000","last_fullsized":"2024-04-15T15:20:01.610534+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:18.761888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610663+0000","last_change":"2024-04-15T15:20:00.614172+0000","last_active":"2024-04-15T15:20:01.610663+0000","last_peered":"2024-04-15T15:20:01.610663+0000","last_clean":"2024-04-15T15:20:01.610663+0000","last_became_active":"2024-04-15T15:20:00.613873+0000","last_became_peered":"2024-04-15T15:20:00.613873+0000","last_unstale":"2024-04-15T15:20:01.610663+0000","last_undegraded":"2024-04-15T15:20:01.610663+0000","last_fullsized":"2024-04-15T15:20:01.610663+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:20:29.604777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620480+0000","last_change":"2024-04-15T15:19:59.571252+0000","last_active":"2024-04-15T15:20:01.620480+0000","last_peered":"2024-04-15T15:20:01.620480+0000","last_clean":"2024-04-15T15:20:01.620480+0000","last_became_active":"2024-04-15T15:19:59.570943+0000","last_became_peered":"2024-04-15T15:19:59.570943+0000","last_unstale":"2024-04-15T15:20:01.620480+0000","last_undegraded":"2024-04-15T15:20:01.620480+0000","last_fullsized":"2024-04-15T15:20:01.620480+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:14:50.511195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904476+0000","last_change":"2024-04-15T15:20:00.616384+0000","last_active":"2024-04-15T15:20:01.904476+0000","last_peered":"2024-04-15T15:20:01.904476+0000","last_clean":"2024-04-15T15:20:01.904476+0000","last_became_active":"2024-04-15T15:20:00.616216+0000","last_became_peered":"2024-04-15T15:20:00.616216+0000","last_unstale":"2024-04-15T15:20:01.904476+0000","last_undegraded":"2024-04-15T15:20:01.904476+0000","last_fullsized":"2024-04-15T15:20:01.904476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:39:11.254906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906546+0000","last_change":"2024-04-15T15:19:59.558193+0000","last_active":"2024-04-15T15:20:01.906546+0000","last_peered":"2024-04-15T15:20:01.906546+0000","last_clean":"2024-04-15T15:20:01.906546+0000","last_became_active":"2024-04-15T15:19:59.557897+0000","last_became_peered":"2024-04-15T15:19:59.557897+0000","last_unstale":"2024-04-15T15:20:01.906546+0000","last_undegraded":"2024-04-15T15:20:01.906546+0000","last_fullsized":"2024-04-15T15:20:01.906546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:26:18.725082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.625003+0000","last_change":"2024-04-15T15:20:00.611926+0000","last_active":"2024-04-15T15:20:01.625003+0000","last_peered":"2024-04-15T15:20:01.625003+0000","last_clean":"2024-04-15T15:20:01.625003+0000","last_became_active":"2024-04-15T15:20:00.611607+0000","last_became_peered":"2024-04-15T15:20:00.611607+0000","last_unstale":"2024-04-15T15:20:01.625003+0000","last_undegraded":"2024-04-15T15:20:01.625003+0000","last_fullsized":"2024-04-15T15:20:01.625003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:17:36.203246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904132+0000","last_change":"2024-04-15T15:19:59.564020+0000","last_active":"2024-04-15T15:20:01.904132+0000","last_peered":"2024-04-15T15:20:01.904132+0000","last_clean":"2024-04-15T15:20:01.904132+0000","last_became_active":"2024-04-15T15:19:59.563731+0000","last_became_peered":"2024-04-15T15:19:59.563731+0000","last_unstale":"2024-04-15T15:20:01.904132+0000","last_undegraded":"2024-04-15T15:20:01.904132+0000","last_fullsized":"2024-04-15T15:20:01.904132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:41:08.033269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.607665+0000","last_change":"2024-04-15T15:20:00.612268+0000","last_active":"2024-04-15T15:20:01.607665+0000","last_peered":"2024-04-15T15:20:01.607665+0000","last_clean":"2024-04-15T15:20:01.607665+0000","last_became_active":"2024-04-15T15:20:00.611977+0000","last_became_peered":"2024-04-15T15:20:00.611977+0000","last_unstale":"2024-04-15T15:20:01.607665+0000","last_undegraded":"2024-04-15T15:20:01.607665+0000","last_fullsized":"2024-04-15T15:20:01.607665+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:02:52.890702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904701+0000","last_change":"2024-04-15T15:19:59.553449+0000","last_active":"2024-04-15T15:20:01.904701+0000","last_peered":"2024-04-15T15:20:01.904701+0000","last_clean":"2024-04-15T15:20:01.904701+0000","last_became_active":"2024-04-15T15:19:59.552881+0000","last_became_peered":"2024-04-15T15:19:59.552881+0000","last_unstale":"2024-04-15T15:20:01.904701+0000","last_undegraded":"2024-04-15T15:20:01.904701+0000","last_fullsized":"2024-04-15T15:20:01.904701+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:34:35.996734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609204+0000","last_change":"2024-04-15T15:20:00.605473+0000","last_active":"2024-04-15T15:20:01.609204+0000","last_peered":"2024-04-15T15:20:01.609204+0000","last_clean":"2024-04-15T15:20:01.609204+0000","last_became_active":"2024-04-15T15:20:00.605208+0000","last_became_peered":"2024-04-15T15:20:00.605208+0000","last_unstale":"2024-04-15T15:20:01.609204+0000","last_undegraded":"2024-04-15T15:20:01.609204+0000","last_fullsized":"2024-04-15T15:20:01.609204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:50:34.114889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904894+0000","last_change":"2024-04-15T15:19:59.549691+0000","last_active":"2024-04-15T15:20:01.904894+0000","last_peered":"2024-04-15T15:20:01.904894+0000","last_clean":"2024-04-15T15:20:01.904894+0000","last_became_active":"2024-04-15T15:19:59.549390+0000","last_became_peered":"2024-04-15T15:19:59.549390+0000","last_unstale":"2024-04-15T15:20:01.904894+0000","last_undegraded":"2024-04-15T15:20:01.904894+0000","last_fullsized":"2024-04-15T15:20:01.904894+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:11:03.397227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904149+0000","last_change":"2024-04-15T15:20:00.609329+0000","last_active":"2024-04-15T15:20:01.904149+0000","last_peered":"2024-04-15T15:20:01.904149+0000","last_clean":"2024-04-15T15:20:01.904149+0000","last_became_active":"2024-04-15T15:20:00.608999+0000","last_became_peered":"2024-04-15T15:20:00.608999+0000","last_unstale":"2024-04-15T15:20:01.904149+0000","last_undegraded":"2024-04-15T15:20:01.904149+0000","last_fullsized":"2024-04-15T15:20:01.904149+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:48:36.275786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905037+0000","last_change":"2024-04-15T15:19:59.555002+0000","last_active":"2024-04-15T15:20:01.905037+0000","last_peered":"2024-04-15T15:20:01.905037+0000","last_clean":"2024-04-15T15:20:01.905037+0000","last_became_active":"2024-04-15T15:19:59.554675+0000","last_became_peered":"2024-04-15T15:19:59.554675+0000","last_unstale":"2024-04-15T15:20:01.905037+0000","last_undegraded":"2024-04-15T15:20:01.905037+0000","last_fullsized":"2024-04-15T15:20:01.905037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:25.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624121+0000","last_change":"2024-04-15T15:20:00.616313+0000","last_active":"2024-04-15T15:20:01.624121+0000","last_peered":"2024-04-15T15:20:01.624121+0000","last_clean":"2024-04-15T15:20:01.624121+0000","last_became_active":"2024-04-15T15:20:00.614709+0000","last_became_peered":"2024-04-15T15:20:00.614709+0000","last_unstale":"2024-04-15T15:20:01.624121+0000","last_undegraded":"2024-04-15T15:20:01.624121+0000","last_fullsized":"2024-04-15T15:20:01.624121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:47:27.051203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904230+0000","last_change":"2024-04-15T15:19:59.557469+0000","last_active":"2024-04-15T15:20:01.904230+0000","last_peered":"2024-04-15T15:20:01.904230+0000","last_clean":"2024-04-15T15:20:01.904230+0000","last_became_active":"2024-04-15T15:19:59.556245+0000","last_became_peered":"2024-04-15T15:19:59.556245+0000","last_unstale":"2024-04-15T15:20:01.904230+0000","last_undegraded":"2024-04-15T15:20:01.904230+0000","last_fullsized":"2024-04-15T15:20:01.904230+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:55:45.235539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608734+0000","last_change":"2024-04-15T15:20:00.616837+0000","last_active":"2024-04-15T15:20:01.608734+0000","last_peered":"2024-04-15T15:20:01.608734+0000","last_clean":"2024-04-15T15:20:01.608734+0000","last_became_active":"2024-04-15T15:20:00.615232+0000","last_became_peered":"2024-04-15T15:20:00.615232+0000","last_unstale":"2024-04-15T15:20:01.608734+0000","last_undegraded":"2024-04-15T15:20:01.608734+0000","last_fullsized":"2024-04-15T15:20:01.608734+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:02:52.890702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906891+0000","last_change":"2024-04-15T15:19:59.566367+0000","last_active":"2024-04-15T15:20:01.906891+0000","last_peered":"2024-04-15T15:20:01.906891+0000","last_clean":"2024-04-15T15:20:01.906891+0000","last_became_active":"2024-04-15T15:19:59.566080+0000","last_became_peered":"2024-04-15T15:19:59.566080+0000","last_unstale":"2024-04-15T15:20:01.906891+0000","last_undegraded":"2024-04-15T15:20:01.906891+0000","last_fullsized":"2024-04-15T15:20:01.906891+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:55:13.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906949+0000","last_change":"2024-04-15T15:20:00.621744+0000","last_active":"2024-04-15T15:20:01.906949+0000","last_peered":"2024-04-15T15:20:01.906949+0000","last_clean":"2024-04-15T15:20:01.906949+0000","last_became_active":"2024-04-15T15:20:00.621464+0000","last_became_peered":"2024-04-15T15:20:00.621464+0000","last_unstale":"2024-04-15T15:20:01.906949+0000","last_undegraded":"2024-04-15T15:20:01.906949+0000","last_fullsized":"2024-04-15T15:20:01.906949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:44:14.986046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620602+0000","last_change":"2024-04-15T15:19:59.557592+0000","last_active":"2024-04-15T15:20:01.620602+0000","last_peered":"2024-04-15T15:20:01.620602+0000","last_clean":"2024-04-15T15:20:01.620602+0000","last_became_active":"2024-04-15T15:19:59.557267+0000","last_became_peered":"2024-04-15T15:19:59.557267+0000","last_unstale":"2024-04-15T15:20:01.620602+0000","last_undegraded":"2024-04-15T15:20:01.620602+0000","last_fullsized":"2024-04-15T15:20:01.620602+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:07:49.627900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906127+0000","last_change":"2024-04-15T15:20:00.608598+0000","last_active":"2024-04-15T15:20:01.906127+0000","last_peered":"2024-04-15T15:20:01.906127+0000","last_clean":"2024-04-15T15:20:01.906127+0000","last_became_active":"2024-04-15T15:20:00.608275+0000","last_became_peered":"2024-04-15T15:20:00.608275+0000","last_unstale":"2024-04-15T15:20:01.906127+0000","last_undegraded":"2024-04-15T15:20:01.906127+0000","last_fullsized":"2024-04-15T15:20:01.906127+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:36:47.757839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608437+0000","last_change":"2024-04-15T15:19:59.566469+0000","last_active":"2024-04-15T15:20:01.608437+0000","last_peered":"2024-04-15T15:20:01.608437+0000","last_clean":"2024-04-15T15:20:01.608437+0000","last_became_active":"2024-04-15T15:19:59.566154+0000","last_became_peered":"2024-04-15T15:19:59.566154+0000","last_unstale":"2024-04-15T15:20:01.608437+0000","last_undegraded":"2024-04-15T15:20:01.608437+0000","last_fullsized":"2024-04-15T15:20:01.608437+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:28:56.698937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608101+0000","last_change":"2024-04-15T15:20:00.611802+0000","last_active":"2024-04-15T15:20:01.608101+0000","last_peered":"2024-04-15T15:20:01.608101+0000","last_clean":"2024-04-15T15:20:01.608101+0000","last_became_active":"2024-04-15T15:20:00.611108+0000","last_became_peered":"2024-04-15T15:20:00.611108+0000","last_unstale":"2024-04-15T15:20:01.608101+0000","last_undegraded":"2024-04-15T15:20:01.608101+0000","last_fullsized":"2024-04-15T15:20:01.608101+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:01:26.510445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906227+0000","last_change":"2024-04-15T15:19:59.559002+0000","last_active":"2024-04-15T15:20:01.906227+0000","last_peered":"2024-04-15T15:20:01.906227+0000","last_clean":"2024-04-15T15:20:01.906227+0000","last_became_active":"2024-04-15T15:19:59.558721+0000","last_became_peered":"2024-04-15T15:19:59.558721+0000","last_unstale":"2024-04-15T15:20:01.906227+0000","last_undegraded":"2024-04-15T15:20:01.906227+0000","last_fullsized":"2024-04-15T15:20:01.906227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:20:28.599563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624263+0000","last_change":"2024-04-15T15:20:00.610191+0000","last_active":"2024-04-15T15:20:01.624263+0000","last_peered":"2024-04-15T15:20:01.624263+0000","last_clean":"2024-04-15T15:20:01.624263+0000","last_became_active":"2024-04-15T15:20:00.609856+0000","last_became_peered":"2024-04-15T15:20:00.609856+0000","last_unstale":"2024-04-15T15:20:01.624263+0000","last_undegraded":"2024-04-15T15:20:01.624263+0000","last_fullsized":"2024-04-15T15:20:01.624263+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:04:28.710555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905009+0000","last_change":"2024-04-15T15:19:59.558430+0000","last_active":"2024-04-15T15:20:01.905009+0000","last_peered":"2024-04-15T15:20:01.905009+0000","last_clean":"2024-04-15T15:20:01.905009+0000","last_became_active":"2024-04-15T15:19:59.557976+0000","last_became_peered":"2024-04-15T15:19:59.557976+0000","last_unstale":"2024-04-15T15:20:01.905009+0000","last_undegraded":"2024-04-15T15:20:01.905009+0000","last_fullsized":"2024-04-15T15:20:01.905009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:46:18.671155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906118+0000","last_change":"2024-04-15T15:20:00.617234+0000","last_active":"2024-04-15T15:20:01.906118+0000","last_peered":"2024-04-15T15:20:01.906118+0000","last_clean":"2024-04-15T15:20:01.906118+0000","last_became_active":"2024-04-15T15:20:00.616888+0000","last_became_peered":"2024-04-15T15:20:00.616888+0000","last_unstale":"2024-04-15T15:20:01.906118+0000","last_undegraded":"2024-04-15T15:20:01.906118+0000","last_fullsized":"2024-04-15T15:20:01.906118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:50:34.114889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624403+0000","last_change":"2024-04-15T15:19:59.556664+0000","last_active":"2024-04-15T15:20:01.624403+0000","last_peered":"2024-04-15T15:20:01.624403+0000","last_clean":"2024-04-15T15:20:01.624403+0000","last_became_active":"2024-04-15T15:19:59.556350+0000","last_became_peered":"2024-04-15T15:19:59.556350+0000","last_unstale":"2024-04-15T15:20:01.624403+0000","last_undegraded":"2024-04-15T15:20:01.624403+0000","last_fullsized":"2024-04-15T15:20:01.624403+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:14:47.709963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.605395+0000","last_change":"2024-04-15T15:20:00.612658+0000","last_active":"2024-04-15T15:20:01.605395+0000","last_peered":"2024-04-15T15:20:01.605395+0000","last_clean":"2024-04-15T15:20:01.605395+0000","last_became_active":"2024-04-15T15:20:00.612422+0000","last_became_peered":"2024-04-15T15:20:00.612422+0000","last_unstale":"2024-04-15T15:20:01.605395+0000","last_undegraded":"2024-04-15T15:20:01.605395+0000","last_fullsized":"2024-04-15T15:20:01.605395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:38:12.140232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905149+0000","last_change":"2024-04-15T15:19:59.567169+0000","last_active":"2024-04-15T15:20:01.905149+0000","last_peered":"2024-04-15T15:20:01.905149+0000","last_clean":"2024-04-15T15:20:01.905149+0000","last_became_active":"2024-04-15T15:19:59.566906+0000","last_became_peered":"2024-04-15T15:19:59.566906+0000","last_unstale":"2024-04-15T15:20:01.905149+0000","last_undegraded":"2024-04-15T15:20:01.905149+0000","last_fullsized":"2024-04-15T15:20:01.905149+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:51:03.017053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609691+0000","last_change":"2024-04-15T15:20:00.613749+0000","last_active":"2024-04-15T15:20:01.609691+0000","last_peered":"2024-04-15T15:20:01.609691+0000","last_clean":"2024-04-15T15:20:01.609691+0000","last_became_active":"2024-04-15T15:20:00.613409+0000","last_became_peered":"2024-04-15T15:20:00.613409+0000","last_unstale":"2024-04-15T15:20:01.609691+0000","last_undegraded":"2024-04-15T15:20:01.609691+0000","last_fullsized":"2024-04-15T15:20:01.609691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:26.988652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905270+0000","last_change":"2024-04-15T15:19:59.558986+0000","last_active":"2024-04-15T15:20:01.905270+0000","last_peered":"2024-04-15T15:20:01.905270+0000","last_clean":"2024-04-15T15:20:01.905270+0000","last_became_active":"2024-04-15T15:19:59.557861+0000","last_became_peered":"2024-04-15T15:19:59.557861+0000","last_unstale":"2024-04-15T15:20:01.905270+0000","last_undegraded":"2024-04-15T15:20:01.905270+0000","last_fullsized":"2024-04-15T15:20:01.905270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:28:56.698937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904955+0000","last_change":"2024-04-15T15:20:00.602513+0000","last_active":"2024-04-15T15:20:01.904955+0000","last_peered":"2024-04-15T15:20:01.904955+0000","last_clean":"2024-04-15T15:20:01.904955+0000","last_became_active":"2024-04-15T15:20:00.601487+0000","last_became_peered":"2024-04-15T15:20:00.601487+0000","last_unstale":"2024-04-15T15:20:01.904955+0000","last_undegraded":"2024-04-15T15:20:01.904955+0000","last_fullsized":"2024-04-15T15:20:01.904955+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:50:35.288023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.625024+0000","last_change":"2024-04-15T15:19:59.569146+0000","last_active":"2024-04-15T15:20:01.625024+0000","last_peered":"2024-04-15T15:20:01.625024+0000","last_clean":"2024-04-15T15:20:01.625024+0000","last_became_active":"2024-04-15T15:19:59.567258+0000","last_became_peered":"2024-04-15T15:19:59.567258+0000","last_unstale":"2024-04-15T15:20:01.625024+0000","last_undegraded":"2024-04-15T15:20:01.625024+0000","last_fullsized":"2024-04-15T15:20:01.625024+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:07:49.627900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609754+0000","last_change":"2024-04-15T15:20:00.597695+0000","last_active":"2024-04-15T15:20:01.609754+0000","last_peered":"2024-04-15T15:20:01.609754+0000","last_clean":"2024-04-15T15:20:01.609754+0000","last_became_active":"2024-04-15T15:20:00.597397+0000","last_became_peered":"2024-04-15T15:20:00.597397+0000","last_unstale":"2024-04-15T15:20:01.609754+0000","last_undegraded":"2024-04-15T15:20:01.609754+0000","last_fullsized":"2024-04-15T15:20:01.609754+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:36:47.757839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904950+0000","last_change":"2024-04-15T15:19:59.558312+0000","last_active":"2024-04-15T15:20:01.904950+0000","last_peered":"2024-04-15T15:20:01.904950+0000","last_clean":"2024-04-15T15:20:01.904950+0000","last_became_active":"2024-04-15T15:19:59.557829+0000","last_became_peered":"2024-04-15T15:19:59.557829+0000","last_unstale":"2024-04-15T15:20:01.904950+0000","last_undegraded":"2024-04-15T15:20:01.904950+0000","last_fullsized":"2024-04-15T15:20:01.904950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:46:37.671572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624442+0000","last_change":"2024-04-15T15:20:00.611135+0000","last_active":"2024-04-15T15:20:01.624442+0000","last_peered":"2024-04-15T15:20:01.624442+0000","last_clean":"2024-04-15T15:20:01.624442+0000","last_became_active":"2024-04-15T15:20:00.610872+0000","last_became_peered":"2024-04-15T15:20:00.610872+0000","last_unstale":"2024-04-15T15:20:01.624442+0000","last_undegraded":"2024-04-15T15:20:01.624442+0000","last_fullsized":"2024-04-15T15:20:01.624442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:01:26.510445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904901+0000","last_change":"2024-04-15T15:19:59.564696+0000","last_active":"2024-04-15T15:20:01.904901+0000","last_peered":"2024-04-15T15:20:01.904901+0000","last_clean":"2024-04-15T15:20:01.904901+0000","last_became_active":"2024-04-15T15:19:59.562001+0000","last_became_peered":"2024-04-15T15:19:59.562001+0000","last_unstale":"2024-04-15T15:20:01.904901+0000","last_undegraded":"2024-04-15T15:20:01.904901+0000","last_fullsized":"2024-04-15T15:20:01.904901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:06:05.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904272+0000","last_change":"2024-04-15T15:20:00.624563+0000","last_active":"2024-04-15T15:20:01.904272+0000","last_peered":"2024-04-15T15:20:01.904272+0000","last_clean":"2024-04-15T15:20:01.904272+0000","last_became_active":"2024-04-15T15:20:00.624235+0000","last_became_peered":"2024-04-15T15:20:00.624235+0000","last_unstale":"2024-04-15T15:20:01.904272+0000","last_undegraded":"2024-04-15T15:20:01.904272+0000","last_fullsized":"2024-04-15T15:20:01.904272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:57:21.317994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904354+0000","last_change":"2024-04-15T15:19:59.560819+0000","last_active":"2024-04-15T15:20:01.904354+0000","last_peered":"2024-04-15T15:20:01.904354+0000","last_clean":"2024-04-15T15:20:01.904354+0000","last_became_active":"2024-04-15T15:19:59.560471+0000","last_became_peered":"2024-04-15T15:19:59.560471+0000","last_unstale":"2024-04-15T15:20:01.904354+0000","last_undegraded":"2024-04-15T15:20:01.904354+0000","last_fullsized":"2024-04-15T15:20:01.904354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:58:32.114685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903548+0000","last_change":"2024-04-15T15:20:00.626744+0000","last_active":"2024-04-15T15:20:01.903548+0000","last_peered":"2024-04-15T15:20:01.903548+0000","last_clean":"2024-04-15T15:20:01.903548+0000","last_became_active":"2024-04-15T15:20:00.621491+0000","last_became_peered":"2024-04-15T15:20:00.621491+0000","last_unstale":"2024-04-15T15:20:01.903548+0000","last_undegraded":"2024-04-15T15:20:01.903548+0000","last_fullsized":"2024-04-15T15:20:01.903548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:53:54.386402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620531+0000","last_change":"2024-04-15T15:19:59.568281+0000","last_active":"2024-04-15T15:20:01.620531+0000","last_peered":"2024-04-15T15:20:01.620531+0000","last_clean":"2024-04-15T15:20:01.620531+0000","last_became_active":"2024-04-15T15:19:59.567805+0000","last_became_peered":"2024-04-15T15:19:59.567805+0000","last_unstale":"2024-04-15T15:20:01.620531+0000","last_undegraded":"2024-04-15T15:20:01.620531+0000","last_fullsized":"2024-04-15T15:20:01.620531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:57:20.312781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903601+0000","last_change":"2024-04-15T15:20:00.628107+0000","last_active":"2024-04-15T15:20:01.903601+0000","last_peered":"2024-04-15T15:20:01.903601+0000","last_clean":"2024-04-15T15:20:01.903601+0000","last_became_active":"2024-04-15T15:20:00.627798+0000","last_became_peered":"2024-04-15T15:20:00.627798+0000","last_unstale":"2024-04-15T15:20:01.903601+0000","last_undegraded":"2024-04-15T15:20:01.903601+0000","last_fullsized":"2024-04-15T15:20:01.903601+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:19.767102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610052+0000","last_change":"2024-04-15T15:19:59.564372+0000","last_active":"2024-04-15T15:20:01.610052+0000","last_peered":"2024-04-15T15:20:01.610052+0000","last_clean":"2024-04-15T15:20:01.610052+0000","last_became_active":"2024-04-15T15:19:59.564094+0000","last_became_peered":"2024-04-15T15:19:59.564094+0000","last_unstale":"2024-04-15T15:20:01.610052+0000","last_undegraded":"2024-04-15T15:20:01.610052+0000","last_fullsized":"2024-04-15T15:20:01.610052+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:26:21.787578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903392+0000","last_change":"2024-04-15T15:20:00.626889+0000","last_active":"2024-04-15T15:20:01.903392+0000","last_peered":"2024-04-15T15:20:01.903392+0000","last_clean":"2024-04-15T15:20:01.903392+0000","last_became_active":"2024-04-15T15:20:00.626572+0000","last_became_peered":"2024-04-15T15:20:00.626572+0000","last_unstale":"2024-04-15T15:20:01.903392+0000","last_undegraded":"2024-04-15T15:20:01.903392+0000","last_fullsized":"2024-04-15T15:20:01.903392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:44:47.893636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903969+0000","last_change":"2024-04-15T15:19:59.565493+0000","last_active":"2024-04-15T15:20:01.903969+0000","last_peered":"2024-04-15T15:20:01.903969+0000","last_clean":"2024-04-15T15:20:01.903969+0000","last_became_active":"2024-04-15T15:19:59.565208+0000","last_became_peered":"2024-04-15T15:19:59.565208+0000","last_unstale":"2024-04-15T15:20:01.903969+0000","last_undegraded":"2024-04-15T15:20:01.903969+0000","last_fullsized":"2024-04-15T15:20:01.903969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:52.079334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.605031+0000","last_change":"2024-04-15T15:20:00.609949+0000","last_active":"2024-04-15T15:20:01.605031+0000","last_peered":"2024-04-15T15:20:01.605031+0000","last_clean":"2024-04-15T15:20:01.605031+0000","last_became_active":"2024-04-15T15:20:00.609511+0000","last_became_peered":"2024-04-15T15:20:00.609511+0000","last_unstale":"2024-04-15T15:20:01.605031+0000","last_undegraded":"2024-04-15T15:20:01.605031+0000","last_fullsized":"2024-04-15T15:20:01.605031+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:00:21.659268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.607589+0000","last_change":"2024-04-15T15:19:59.566727+0000","last_active":"2024-04-15T15:20:01.607589+0000","last_peered":"2024-04-15T15:20:01.607589+0000","last_clean":"2024-04-15T15:20:01.607589+0000","last_became_active":"2024-04-15T15:19:59.566169+0000","last_became_peered":"2024-04-15T15:19:59.566169+0000","last_unstale":"2024-04-15T15:20:01.607589+0000","last_undegraded":"2024-04-15T15:20:01.607589+0000","last_fullsized":"2024-04-15T15:20:01.607589+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:25.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904431+0000","last_change":"2024-04-15T15:20:00.624647+0000","last_active":"2024-04-15T15:20:01.904431+0000","last_peered":"2024-04-15T15:20:01.904431+0000","last_clean":"2024-04-15T15:20:01.904431+0000","last_became_active":"2024-04-15T15:20:00.624327+0000","last_became_peered":"2024-04-15T15:20:00.624327+0000","last_unstale":"2024-04-15T15:20:01.904431+0000","last_undegraded":"2024-04-15T15:20:01.904431+0000","last_fullsized":"2024-04-15T15:20:01.904431+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:14:51.516409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.625034+0000","last_change":"2024-04-15T15:19:59.570791+0000","last_active":"2024-04-15T15:20:01.625034+0000","last_peered":"2024-04-15T15:20:01.625034+0000","last_clean":"2024-04-15T15:20:01.625034+0000","last_became_active":"2024-04-15T15:19:59.570453+0000","last_became_peered":"2024-04-15T15:19:59.570453+0000","last_unstale":"2024-04-15T15:20:01.625034+0000","last_undegraded":"2024-04-15T15:20:01.625034+0000","last_fullsized":"2024-04-15T15:20:01.625034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:14:42.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610556+0000","last_change":"2024-04-15T15:20:00.614557+0000","last_active":"2024-04-15T15:20:01.610556+0000","last_peered":"2024-04-15T15:20:01.610556+0000","last_clean":"2024-04-15T15:20:01.610556+0000","last_became_active":"2024-04-15T15:20:00.614289+0000","last_became_peered":"2024-04-15T15:20:00.614289+0000","last_unstale":"2024-04-15T15:20:01.610556+0000","last_undegraded":"2024-04-15T15:20:01.610556+0000","last_fullsized":"2024-04-15T15:20:01.610556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:53.187316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.623360+0000","last_change":"2024-04-15T15:19:59.565639+0000","last_active":"2024-04-15T15:20:01.623360+0000","last_peered":"2024-04-15T15:20:01.623360+0000","last_clean":"2024-04-15T15:20:01.623360+0000","last_became_active":"2024-04-15T15:19:59.565287+0000","last_became_peered":"2024-04-15T15:19:59.565287+0000","last_unstale":"2024-04-15T15:20:01.623360+0000","last_undegraded":"2024-04-15T15:20:01.623360+0000","last_fullsized":"2024-04-15T15:20:01.623360+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:14:42.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620514+0000","last_change":"2024-04-15T15:20:00.605815+0000","last_active":"2024-04-15T15:20:01.620514+0000","last_peered":"2024-04-15T15:20:01.620514+0000","last_clean":"2024-04-15T15:20:01.620514+0000","last_became_active":"2024-04-15T15:20:00.605350+0000","last_became_peered":"2024-04-15T15:20:00.605350+0000","last_unstale":"2024-04-15T15:20:01.620514+0000","last_undegraded":"2024-04-15T15:20:01.620514+0000","last_fullsized":"2024-04-15T15:20:01.620514+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:26:19.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624667+0000","last_change":"2024-04-15T15:19:59.560517+0000","last_active":"2024-04-15T15:20:01.624667+0000","last_peered":"2024-04-15T15:20:01.624667+0000","last_clean":"2024-04-15T15:20:01.624667+0000","last_became_active":"2024-04-15T15:19:59.560021+0000","last_became_peered":"2024-04-15T15:19:59.560021+0000","last_unstale":"2024-04-15T15:20:01.624667+0000","last_undegraded":"2024-04-15T15:20:01.624667+0000","last_fullsized":"2024-04-15T15:20:01.624667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:14:50.511195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609669+0000","last_change":"2024-04-15T15:20:00.619344+0000","last_active":"2024-04-15T15:20:01.609669+0000","last_peered":"2024-04-15T15:20:01.609669+0000","last_clean":"2024-04-15T15:20:01.609669+0000","last_became_active":"2024-04-15T15:20:00.619057+0000","last_became_peered":"2024-04-15T15:20:00.619057+0000","last_unstale":"2024-04-15T15:20:01.609669+0000","last_undegraded":"2024-04-15T15:20:01.609669+0000","last_fullsized":"2024-04-15T15:20:01.609669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:55:31.770386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620710+0000","last_change":"2024-04-15T15:19:59.560841+0000","last_active":"2024-04-15T15:20:01.620710+0000","last_peered":"2024-04-15T15:20:01.620710+0000","last_clean":"2024-04-15T15:20:01.620710+0000","last_became_active":"2024-04-15T15:19:59.560515+0000","last_became_peered":"2024-04-15T15:19:59.560515+0000","last_unstale":"2024-04-15T15:20:01.620710+0000","last_undegraded":"2024-04-15T15:20:01.620710+0000","last_fullsized":"2024-04-15T15:20:01.620710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:14:47.709963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620585+0000","last_change":"2024-04-15T15:20:00.613259+0000","last_active":"2024-04-15T15:20:01.620585+0000","last_peered":"2024-04-15T15:20:01.620585+0000","last_clean":"2024-04-15T15:20:01.620585+0000","last_became_active":"2024-04-15T15:20:00.612923+0000","last_became_peered":"2024-04-15T15:20:00.612923+0000","last_unstale":"2024-04-15T15:20:01.620585+0000","last_undegraded":"2024-04-15T15:20:01.620585+0000","last_fullsized":"2024-04-15T15:20:01.620585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:55:14.237241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624544+0000","last_change":"2024-04-15T15:19:59.569507+0000","last_active":"2024-04-15T15:20:01.624544+0000","last_peered":"2024-04-15T15:20:01.624544+0000","last_clean":"2024-04-15T15:20:01.624544+0000","last_became_active":"2024-04-15T15:19:59.568995+0000","last_became_peered":"2024-04-15T15:19:59.568995+0000","last_unstale":"2024-04-15T15:20:01.624544+0000","last_undegraded":"2024-04-15T15:20:01.624544+0000","last_fullsized":"2024-04-15T15:20:01.624544+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:32:06.507059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904841+0000","last_change":"2024-04-15T15:20:00.612377+0000","last_active":"2024-04-15T15:20:01.904841+0000","last_peered":"2024-04-15T15:20:01.904841+0000","last_clean":"2024-04-15T15:20:01.904841+0000","last_became_active":"2024-04-15T15:20:00.612064+0000","last_became_peered":"2024-04-15T15:20:00.612064+0000","last_unstale":"2024-04-15T15:20:01.904841+0000","last_undegraded":"2024-04-15T15:20:01.904841+0000","last_fullsized":"2024-04-15T15:20:01.904841+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:17:36.203246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904268+0000","last_change":"2024-04-15T15:19:59.562470+0000","last_active":"2024-04-15T15:20:01.904268+0000","last_peered":"2024-04-15T15:20:01.904268+0000","last_clean":"2024-04-15T15:20:01.904268+0000","last_became_active":"2024-04-15T15:19:59.560405+0000","last_became_peered":"2024-04-15T15:19:59.560405+0000","last_unstale":"2024-04-15T15:20:01.904268+0000","last_undegraded":"2024-04-15T15:20:01.904268+0000","last_fullsized":"2024-04-15T15:20:01.904268+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:28:01.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903985+0000","last_change":"2024-04-15T15:20:00.615924+0000","last_active":"2024-04-15T15:20:01.903985+0000","last_peered":"2024-04-15T15:20:01.903985+0000","last_clean":"2024-04-15T15:20:01.903985+0000","last_became_active":"2024-04-15T15:20:00.615644+0000","last_became_peered":"2024-04-15T15:20:00.615644+0000","last_unstale":"2024-04-15T15:20:01.903985+0000","last_undegraded":"2024-04-15T15:20:01.903985+0000","last_fullsized":"2024-04-15T15:20:01.903985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:44:57.445057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904978+0000","last_change":"2024-04-15T15:19:59.570049+0000","last_active":"2024-04-15T15:20:01.904978+0000","last_peered":"2024-04-15T15:20:01.904978+0000","last_clean":"2024-04-15T15:20:01.904978+0000","last_became_active":"2024-04-15T15:19:59.568744+0000","last_became_peered":"2024-04-15T15:19:59.568744+0000","last_unstale":"2024-04-15T15:20:01.904978+0000","last_undegraded":"2024-04-15T15:20:01.904978+0000","last_fullsized":"2024-04-15T15:20:01.904978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:55:30.765172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.607013+0000","last_change":"2024-04-15T15:20:00.624541+0000","last_active":"2024-04-15T15:20:01.607013+0000","last_peered":"2024-04-15T15:20:01.607013+0000","last_clean":"2024-04-15T15:20:01.607013+0000","last_became_active":"2024-04-15T15:20:00.624291+0000","last_became_peered":"2024-04-15T15:20:00.624291+0000","last_unstale":"2024-04-15T15:20:01.607013+0000","last_undegraded":"2024-04-15T15:20:01.607013+0000","last_fullsized":"2024-04-15T15:20:01.607013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:50:34.114889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610154+0000","last_change":"2024-04-15T15:19:59.563726+0000","last_active":"2024-04-15T15:20:01.610154+0000","last_peered":"2024-04-15T15:20:01.610154+0000","last_clean":"2024-04-15T15:20:01.610154+0000","last_became_active":"2024-04-15T15:19:59.563395+0000","last_became_peered":"2024-04-15T15:19:59.563395+0000","last_unstale":"2024-04-15T15:20:01.610154+0000","last_undegraded":"2024-04-15T15:20:01.610154+0000","last_fullsized":"2024-04-15T15:20:01.610154+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:43:38.542073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.606369+0000","last_change":"2024-04-15T15:18:41.162887+0000","last_active":"2024-04-15T15:20:01.606369+0000","last_peered":"2024-04-15T15:20:01.606369+0000","last_clean":"2024-04-15T15:20:01.606369+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:20:01.606369+0000","last_undegraded":"2024-04-15T15:20:01.606369+0000","last_fullsized":"2024-04-15T15:20:01.606369+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_clean_scrub_stamp":"2024-04-15T15:18:39.644354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:32:00.047414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":991232,"data_stored":745040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":155198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":155254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":409600,"data_stored":156216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":385024,"data_stored":148352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":409600,"data_stored":156216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":744950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:20:05.122 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:20:05.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.390+0000 7f1c8fe00640 1 Processor -- start 2024-04-15T15:20:05.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.390+0000 7f1c8fe00640 1 -- start start 2024-04-15T15:20:05.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.392+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c9015b480 0x7f1c900cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.392+0000 7f1c8fe00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1c9015b860 0x7f1c900b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.392+0000 7f1c8fe00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9015f2b0 con 0x7f1c900cdde0 2024-04-15T15:20:05.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.392+0000 7f1c8fe00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9015f060 con 0x7f1c9015b860 2024-04-15T15:20:05.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.392+0000 7f1c8fe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9015fec0 con 0x7f1c9015b480 2024-04-15T15:20:05.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.392+0000 7f1c8e000640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1c900cdde0 0x7f1c9006ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:40386/0 (socket says 172.21.15.49:40386) 2024-04-15T15:20:05.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.392+0000 7f1c8e000640 1 -- 172.21.15.49:0/571154528 learned_addr learned my addr 172.21.15.49:0/571154528 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:05.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.393+0000 7f1c8f400640 1 --2- 172.21.15.49:0/571154528 >> v2:172.21.15.107:3300/0 conn(0x7f1c9015b860 0x7f1c900b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.393+0000 7f1c8ea00640 1 --2- 172.21.15.49:0/571154528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c9015b480 0x7f1c900cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.393+0000 7f1c8d600640 1 -- 172.21.15.49:0/571154528 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287321005 0 0) 0x7f1c9015f2b0 con 0x7f1c900cdde0 2024-04-15T15:20:05.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.393+0000 7f1c8d600640 1 -- 172.21.15.49:0/571154528 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c74003520 con 0x7f1c900cdde0 2024-04-15T15:20:05.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.393+0000 7f1c8f400640 1 -- 172.21.15.49:0/571154528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c9015b480 msgr2=0x7f1c900cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:05.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.393+0000 7f1c8f400640 1 --2- 172.21.15.49:0/571154528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c9015b480 0x7f1c900cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.393+0000 7f1c8f400640 1 -- 172.21.15.49:0/571154528 >> v1:172.21.15.49:6789/0 conn(0x7f1c900cdde0 legacy=0x7f1c9006ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.393+0000 7f1c8f400640 1 -- 172.21.15.49:0/571154528 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9015f590 con 0x7f1c9015b860 2024-04-15T15:20:05.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.395+0000 7f1c8f400640 1 --2- 172.21.15.49:0/571154528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9015b860 0x7f1c900b3fc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c9015f1e0 tx=0x7f1c8802be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c587c731944899f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.396+0000 7f1c8d600640 1 -- 172.21.15.49:0/571154528 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88039070 con 0x7f1c9015b860 2024-04-15T15:20:05.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.396+0000 7f1c8d600640 1 -- 172.21.15.49:0/571154528 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c8802cc40 con 0x7f1c9015b860 2024-04-15T15:20:05.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.396+0000 7f1c8d600640 1 -- 172.21.15.49:0/571154528 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88034440 con 0x7f1c9015b860 2024-04-15T15:20:05.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.396+0000 7f1c8fe00640 1 -- 172.21.15.49:0/571154528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9015b860 msgr2=0x7f1c900b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.396+0000 7f1c8fe00640 1 --2- 172.21.15.49:0/571154528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9015b860 0x7f1c900b3fc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c9015f1e0 tx=0x7f1c8802be90 comp rx=0 tx=0).stop 2024-04-15T15:20:05.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.396+0000 7f1c8fe00640 1 -- 172.21.15.49:0/571154528 shutdown_connections 2024-04-15T15:20:05.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.396+0000 7f1c8fe00640 1 --2- 172.21.15.49:0/571154528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9015b860 0x7f1c900b3fc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.396+0000 7f1c8fe00640 1 --2- 172.21.15.49:0/571154528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c9015b480 0x7f1c900cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.396+0000 7f1c8fe00640 1 -- 172.21.15.49:0/571154528 >> 172.21.15.49:0/571154528 conn(0x7f1c9006b930 msgr2=0x7f1c9006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:05.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.397+0000 7f1c8fe00640 1 -- 172.21.15.49:0/571154528 shutdown_connections 2024-04-15T15:20:05.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.397+0000 7f1c8fe00640 1 -- 172.21.15.49:0/571154528 wait complete. 2024-04-15T15:20:05.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.398+0000 7f1c8fe00640 1 Processor -- start 2024-04-15T15:20:05.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.399+0000 7f1c8fe00640 1 -- start start 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.399+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c900cdde0 0x7f1c900b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.399+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c9015b480 0x7f1c9013eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.399+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9015b860 0x7f1c9013f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.399+0000 7f1c8fe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c900cd550 con 0x7f1c900cdde0 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.399+0000 7f1c8fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9015fb50 con 0x7f1c9015b860 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.399+0000 7f1c8fe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9006f4f0 con 0x7f1c9015b480 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.400+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c900cdde0 0x7f1c900b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.400+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c900cdde0 0x7f1c900b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42850/0 (socket says 172.21.15.49:42850) 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.400+0000 7f1c8ea00640 1 -- 172.21.15.49:0/1763960322 learned_addr learned my addr 172.21.15.49:0/1763960322 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.400+0000 7f1c8ea00640 1 -- 172.21.15.49:0/1763960322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c9015b480 msgr2=0x7f1c9013eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.400+0000 7f1c8ea00640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c9015b480 0x7f1c9013eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.400+0000 7f1c8ea00640 1 -- 172.21.15.49:0/1763960322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9015b860 msgr2=0x7f1c9013f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.400+0000 7f1c8ea00640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9015b860 0x7f1c9013f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.400+0000 7f1c8ea00640 1 -- 172.21.15.49:0/1763960322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c90159a50 con 0x7f1c900cdde0 2024-04-15T15:20:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.401+0000 7f1c8ea00640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c900cdde0 0x7f1c900b1890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000e9e0 tx=0x7f1c8000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.401+0000 7f1c87400640 1 -- 172.21.15.49:0/1763960322 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80017070 con 0x7f1c900cdde0 2024-04-15T15:20:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.401+0000 7f1c87400640 1 -- 172.21.15.49:0/1763960322 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c8000c8f0 con 0x7f1c900cdde0 2024-04-15T15:20:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.401+0000 7f1c8fe00640 1 -- 172.21.15.49:0/1763960322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9013cbc0 con 0x7f1c900cdde0 2024-04-15T15:20:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.401+0000 7f1c87400640 1 -- 172.21.15.49:0/1763960322 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000cce0 con 0x7f1c900cdde0 2024-04-15T15:20:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.401+0000 7f1c8fe00640 1 -- 172.21.15.49:0/1763960322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9013d180 con 0x7f1c900cdde0 2024-04-15T15:20:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.402+0000 7f1c8fe00640 1 -- 172.21.15.49:0/1763960322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c90159d30 con 0x7f1c900cdde0 2024-04-15T15:20:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.403+0000 7f1c87400640 1 -- 172.21.15.49:0/1763960322 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c8001a070 con 0x7f1c900cdde0 2024-04-15T15:20:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.404+0000 7f1c87400640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1c7006c8f0 0x7f1c7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.405+0000 7f1c87400640 1 -- 172.21.15.49:0/1763960322 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1c8008be30 con 0x7f1c900cdde0 2024-04-15T15:20:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.405+0000 7f1c8e000640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1c7006c8f0 0x7f1c7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.406+0000 7f1c8e000640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1c7006c8f0 0x7f1c7006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c000f60 tx=0x7f1c7c0078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.409+0000 7f1c87400640 1 -- 172.21.15.49:0/1763960322 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c8005a290 con 0x7f1c900cdde0 2024-04-15T15:20:05.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.611+0000 7f1c8fe00640 1 -- 172.21.15.49:0/1763960322 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c900755c0 con 0x7f1c7006c8f0 2024-04-15T15:20:05.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.619+0000 7f1c87400640 1 -- 172.21.15.49:0/1763960322 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7f1c900755c0 con 0x7f1c7006c8f0 2024-04-15T15:20:05.624 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:05.625 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:20:05.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.624+0000 7f1c8fe00640 1 -- 172.21.15.49:0/1763960322 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1c7006c8f0 msgr2=0x7f1c7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.624+0000 7f1c8fe00640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1c7006c8f0 0x7f1c7006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c000f60 tx=0x7f1c7c0078f0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.624+0000 7f1c8fe00640 1 -- 172.21.15.49:0/1763960322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c900cdde0 msgr2=0x7f1c900b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.624+0000 7f1c8fe00640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c900cdde0 0x7f1c900b1890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000e9e0 tx=0x7f1c8000eea0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.624+0000 7f1c8fe00640 1 -- 172.21.15.49:0/1763960322 shutdown_connections 2024-04-15T15:20:05.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.624+0000 7f1c8fe00640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1c7006c8f0 0x7f1c7006edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.624+0000 7f1c8fe00640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c9015b480 0x7f1c9013eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.624+0000 7f1c8fe00640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9015b860 0x7f1c9013f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.624+0000 7f1c8fe00640 1 --2- 172.21.15.49:0/1763960322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c900cdde0 0x7f1c900b1890 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.625+0000 7f1c8fe00640 1 -- 172.21.15.49:0/1763960322 >> 172.21.15.49:0/1763960322 conn(0x7f1c9006b930 msgr2=0x7f1c900b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:05.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.625+0000 7f1c8fe00640 1 -- 172.21.15.49:0/1763960322 shutdown_connections 2024-04-15T15:20:05.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.625+0000 7f1c8fe00640 1 -- 172.21.15.49:0/1763960322 wait complete. 2024-04-15T15:20:05.645 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-15T15:20:04.796908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237104,"kb_used_data":4272,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737616,"statfs":{"total":773094113280,"available":772851318784,"internally_reserved":0,"allocated":4374528,"data_stored":2409612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002345"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610391+0000","last_change":"2024-04-15T15:20:00.609222+0000","last_active":"2024-04-15T15:20:01.610391+0000","last_peered":"2024-04-15T15:20:01.610391+0000","last_clean":"2024-04-15T15:20:01.610391+0000","last_became_active":"2024-04-15T15:20:00.608895+0000","last_became_peered":"2024-04-15T15:20:00.608895+0000","last_unstale":"2024-04-15T15:20:01.610391+0000","last_undegraded":"2024-04-15T15:20:01.610391+0000","last_fullsized":"2024-04-15T15:20:01.610391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:51:04.022267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906871+0000","last_change":"2024-04-15T15:20:00.622797+0000","last_active":"2024-04-15T15:20:01.906871+0000","last_peered":"2024-04-15T15:20:01.906871+0000","last_clean":"2024-04-15T15:20:01.906871+0000","last_became_active":"2024-04-15T15:20:00.621060+0000","last_became_peered":"2024-04-15T15:20:00.621060+0000","last_unstale":"2024-04-15T15:20:01.906871+0000","last_undegraded":"2024-04-15T15:20:01.906871+0000","last_fullsized":"2024-04-15T15:20:01.906871+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:54.362725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608515+0000","last_change":"2024-04-15T15:20:00.612626+0000","last_active":"2024-04-15T15:20:01.608515+0000","last_peered":"2024-04-15T15:20:01.608515+0000","last_clean":"2024-04-15T15:20:01.608515+0000","last_became_active":"2024-04-15T15:20:00.611057+0000","last_became_peered":"2024-04-15T15:20:00.611057+0000","last_unstale":"2024-04-15T15:20:01.608515+0000","last_undegraded":"2024-04-15T15:20:01.608515+0000","last_fullsized":"2024-04-15T15:20:01.608515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:17:36.203246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624346+0000","last_change":"2024-04-15T15:20:00.621018+0000","last_active":"2024-04-15T15:20:01.624346+0000","last_peered":"2024-04-15T15:20:01.624346+0000","last_clean":"2024-04-15T15:20:01.624346+0000","last_became_active":"2024-04-15T15:20:00.620101+0000","last_became_peered":"2024-04-15T15:20:00.620101+0000","last_unstale":"2024-04-15T15:20:01.624346+0000","last_undegraded":"2024-04-15T15:20:01.624346+0000","last_fullsized":"2024-04-15T15:20:01.624346+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:00:21.659268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903567+0000","last_change":"2024-04-15T15:20:00.622082+0000","last_active":"2024-04-15T15:20:01.903567+0000","last_peered":"2024-04-15T15:20:01.903567+0000","last_clean":"2024-04-15T15:20:01.903567+0000","last_became_active":"2024-04-15T15:20:00.621734+0000","last_became_peered":"2024-04-15T15:20:00.621734+0000","last_unstale":"2024-04-15T15:20:01.903567+0000","last_undegraded":"2024-04-15T15:20:01.903567+0000","last_fullsized":"2024-04-15T15:20:01.903567+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:26:22.792791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905334+0000","last_change":"2024-04-15T15:20:00.612629+0000","last_active":"2024-04-15T15:20:01.905334+0000","last_peered":"2024-04-15T15:20:01.905334+0000","last_clean":"2024-04-15T15:20:01.905334+0000","last_became_active":"2024-04-15T15:20:00.609581+0000","last_became_peered":"2024-04-15T15:20:00.609581+0000","last_unstale":"2024-04-15T15:20:01.905334+0000","last_undegraded":"2024-04-15T15:20:01.905334+0000","last_fullsized":"2024-04-15T15:20:01.905334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:02:52.890702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.605290+0000","last_change":"2024-04-15T15:20:00.609734+0000","last_active":"2024-04-15T15:20:01.605290+0000","last_peered":"2024-04-15T15:20:01.605290+0000","last_clean":"2024-04-15T15:20:01.605290+0000","last_became_active":"2024-04-15T15:20:00.609154+0000","last_became_peered":"2024-04-15T15:20:00.609154+0000","last_unstale":"2024-04-15T15:20:01.605290+0000","last_undegraded":"2024-04-15T15:20:01.605290+0000","last_fullsized":"2024-04-15T15:20:01.605290+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:01:26.510445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.604878+0000","last_change":"2024-04-15T15:20:00.619025+0000","last_active":"2024-04-15T15:20:01.604878+0000","last_peered":"2024-04-15T15:20:01.604878+0000","last_clean":"2024-04-15T15:20:01.604878+0000","last_became_active":"2024-04-15T15:20:00.618572+0000","last_became_peered":"2024-04-15T15:20:00.618572+0000","last_unstale":"2024-04-15T15:20:01.604878+0000","last_undegraded":"2024-04-15T15:20:01.604878+0000","last_fullsized":"2024-04-15T15:20:01.604878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:04:28.710555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905001+0000","last_change":"2024-04-15T15:20:00.617554+0000","last_active":"2024-04-15T15:20:01.905001+0000","last_peered":"2024-04-15T15:20:01.905001+0000","last_clean":"2024-04-15T15:20:01.905001+0000","last_became_active":"2024-04-15T15:20:00.617000+0000","last_became_peered":"2024-04-15T15:20:00.617000+0000","last_unstale":"2024-04-15T15:20:01.905001+0000","last_undegraded":"2024-04-15T15:20:01.905001+0000","last_fullsized":"2024-04-15T15:20:01.905001+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:04:16.834409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624693+0000","last_change":"2024-04-15T15:20:00.624239+0000","last_active":"2024-04-15T15:20:01.624693+0000","last_peered":"2024-04-15T15:20:01.624693+0000","last_clean":"2024-04-15T15:20:01.624693+0000","last_became_active":"2024-04-15T15:20:00.623613+0000","last_became_peered":"2024-04-15T15:20:00.623613+0000","last_unstale":"2024-04-15T15:20:01.624693+0000","last_undegraded":"2024-04-15T15:20:01.624693+0000","last_fullsized":"2024-04-15T15:20:01.624693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:38:12.140232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.607602+0000","last_change":"2024-04-15T15:20:00.610848+0000","last_active":"2024-04-15T15:20:01.607602+0000","last_peered":"2024-04-15T15:20:01.607602+0000","last_clean":"2024-04-15T15:20:01.607602+0000","last_became_active":"2024-04-15T15:20:00.610572+0000","last_became_peered":"2024-04-15T15:20:00.610572+0000","last_unstale":"2024-04-15T15:20:01.607602+0000","last_undegraded":"2024-04-15T15:20:01.607602+0000","last_fullsized":"2024-04-15T15:20:01.607602+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:36:47.757839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608493+0000","last_change":"2024-04-15T15:20:00.610374+0000","last_active":"2024-04-15T15:20:01.608493+0000","last_peered":"2024-04-15T15:20:01.608493+0000","last_clean":"2024-04-15T15:20:01.608493+0000","last_became_active":"2024-04-15T15:20:00.609999+0000","last_became_peered":"2024-04-15T15:20:00.609999+0000","last_unstale":"2024-04-15T15:20:01.608493+0000","last_undegraded":"2024-04-15T15:20:01.608493+0000","last_fullsized":"2024-04-15T15:20:01.608493+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:35:41.539419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903431+0000","last_change":"2024-04-15T15:20:00.617475+0000","last_active":"2024-04-15T15:20:01.903431+0000","last_peered":"2024-04-15T15:20:01.903431+0000","last_clean":"2024-04-15T15:20:01.903431+0000","last_became_active":"2024-04-15T15:20:00.615756+0000","last_became_peered":"2024-04-15T15:20:00.615756+0000","last_unstale":"2024-04-15T15:20:01.903431+0000","last_undegraded":"2024-04-15T15:20:01.903431+0000","last_fullsized":"2024-04-15T15:20:01.903431+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:52:57.430907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905447+0000","last_change":"2024-04-15T15:20:00.628890+0000","last_active":"2024-04-15T15:20:01.905447+0000","last_peered":"2024-04-15T15:20:01.905447+0000","last_clean":"2024-04-15T15:20:01.905447+0000","last_became_active":"2024-04-15T15:20:00.628579+0000","last_became_peered":"2024-04-15T15:20:00.628579+0000","last_unstale":"2024-04-15T15:20:01.905447+0000","last_undegraded":"2024-04-15T15:20:01.905447+0000","last_fullsized":"2024-04-15T15:20:01.905447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:00:29.913810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608265+0000","last_change":"2024-04-15T15:20:00.609953+0000","last_active":"2024-04-15T15:20:01.608265+0000","last_peered":"2024-04-15T15:20:01.608265+0000","last_clean":"2024-04-15T15:20:01.608265+0000","last_became_active":"2024-04-15T15:20:00.609645+0000","last_became_peered":"2024-04-15T15:20:00.609645+0000","last_unstale":"2024-04-15T15:20:01.608265+0000","last_undegraded":"2024-04-15T15:20:01.608265+0000","last_fullsized":"2024-04-15T15:20:01.608265+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:30:16.127825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.606218+0000","last_change":"2024-04-15T15:20:00.616605+0000","last_active":"2024-04-15T15:20:01.606218+0000","last_peered":"2024-04-15T15:20:01.606218+0000","last_clean":"2024-04-15T15:20:01.606218+0000","last_became_active":"2024-04-15T15:20:00.616258+0000","last_became_peered":"2024-04-15T15:20:00.616258+0000","last_unstale":"2024-04-15T15:20:01.606218+0000","last_undegraded":"2024-04-15T15:20:01.606218+0000","last_fullsized":"2024-04-15T15:20:01.606218+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:17:36.203246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.606119+0000","last_change":"2024-04-15T15:20:00.615339+0000","last_active":"2024-04-15T15:20:01.606119+0000","last_peered":"2024-04-15T15:20:01.606119+0000","last_clean":"2024-04-15T15:20:01.606119+0000","last_became_active":"2024-04-15T15:20:00.613420+0000","last_became_peered":"2024-04-15T15:20:00.613420+0000","last_unstale":"2024-04-15T15:20:01.606119+0000","last_undegraded":"2024-04-15T15:20:01.606119+0000","last_fullsized":"2024-04-15T15:20:01.606119+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:35:41.539419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906912+0000","last_change":"2024-04-15T15:20:00.623316+0000","last_active":"2024-04-15T15:20:01.906912+0000","last_peered":"2024-04-15T15:20:01.906912+0000","last_clean":"2024-04-15T15:20:01.906912+0000","last_became_active":"2024-04-15T15:20:00.623048+0000","last_became_peered":"2024-04-15T15:20:00.623048+0000","last_unstale":"2024-04-15T15:20:01.906912+0000","last_undegraded":"2024-04-15T15:20:01.906912+0000","last_fullsized":"2024-04-15T15:20:01.906912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:27.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906162+0000","last_change":"2024-04-15T15:20:00.615947+0000","last_active":"2024-04-15T15:20:01.906162+0000","last_peered":"2024-04-15T15:20:01.906162+0000","last_clean":"2024-04-15T15:20:01.906162+0000","last_became_active":"2024-04-15T15:20:00.615629+0000","last_became_peered":"2024-04-15T15:20:00.615629+0000","last_unstale":"2024-04-15T15:20:01.906162+0000","last_undegraded":"2024-04-15T15:20:01.906162+0000","last_fullsized":"2024-04-15T15:20:01.906162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:53.187316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903746+0000","last_change":"2024-04-15T15:20:00.620795+0000","last_active":"2024-04-15T15:20:01.903746+0000","last_peered":"2024-04-15T15:20:01.903746+0000","last_clean":"2024-04-15T15:20:01.903746+0000","last_became_active":"2024-04-15T15:20:00.620452+0000","last_became_peered":"2024-04-15T15:20:00.620452+0000","last_unstale":"2024-04-15T15:20:01.903746+0000","last_undegraded":"2024-04-15T15:20:01.903746+0000","last_fullsized":"2024-04-15T15:20:01.903746+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:43:39.547287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904403+0000","last_change":"2024-04-15T15:20:00.609970+0000","last_active":"2024-04-15T15:20:01.904403+0000","last_peered":"2024-04-15T15:20:01.904403+0000","last_clean":"2024-04-15T15:20:01.904403+0000","last_became_active":"2024-04-15T15:20:00.609657+0000","last_became_peered":"2024-04-15T15:20:00.609657+0000","last_unstale":"2024-04-15T15:20:01.904403+0000","last_undegraded":"2024-04-15T15:20:01.904403+0000","last_fullsized":"2024-04-15T15:20:01.904403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:34:17.016954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904529+0000","last_change":"2024-04-15T15:20:00.617412+0000","last_active":"2024-04-15T15:20:01.904529+0000","last_peered":"2024-04-15T15:20:01.904529+0000","last_clean":"2024-04-15T15:20:01.904529+0000","last_became_active":"2024-04-15T15:20:00.617083+0000","last_became_peered":"2024-04-15T15:20:00.617083+0000","last_unstale":"2024-04-15T15:20:01.904529+0000","last_undegraded":"2024-04-15T15:20:01.904529+0000","last_fullsized":"2024-04-15T15:20:01.904529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:47:52.358011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620254+0000","last_change":"2024-04-15T15:20:00.601947+0000","last_active":"2024-04-15T15:20:01.620254+0000","last_peered":"2024-04-15T15:20:01.620254+0000","last_clean":"2024-04-15T15:20:01.620254+0000","last_became_active":"2024-04-15T15:20:00.601497+0000","last_became_peered":"2024-04-15T15:20:00.601497+0000","last_unstale":"2024-04-15T15:20:01.620254+0000","last_undegraded":"2024-04-15T15:20:01.620254+0000","last_fullsized":"2024-04-15T15:20:01.620254+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:51:04.022267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608185+0000","last_change":"2024-04-15T15:20:00.614918+0000","last_active":"2024-04-15T15:20:01.608185+0000","last_peered":"2024-04-15T15:20:01.608185+0000","last_clean":"2024-04-15T15:20:01.608185+0000","last_became_active":"2024-04-15T15:20:00.614630+0000","last_became_peered":"2024-04-15T15:20:00.614630+0000","last_unstale":"2024-04-15T15:20:01.608185+0000","last_undegraded":"2024-04-15T15:20:01.608185+0000","last_fullsized":"2024-04-15T15:20:01.608185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:38:12.140232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624909+0000","last_change":"2024-04-15T15:20:00.613167+0000","last_active":"2024-04-15T15:20:01.624909+0000","last_peered":"2024-04-15T15:20:01.624909+0000","last_clean":"2024-04-15T15:20:01.624909+0000","last_became_active":"2024-04-15T15:20:00.612760+0000","last_became_peered":"2024-04-15T15:20:00.612760+0000","last_unstale":"2024-04-15T15:20:01.624909+0000","last_undegraded":"2024-04-15T15:20:01.624909+0000","last_fullsized":"2024-04-15T15:20:01.624909+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:30:16.127825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.907193+0000","last_change":"2024-04-15T15:20:00.612735+0000","last_active":"2024-04-15T15:20:01.907193+0000","last_peered":"2024-04-15T15:20:01.907193+0000","last_clean":"2024-04-15T15:20:01.907193+0000","last_became_active":"2024-04-15T15:20:00.612258+0000","last_became_peered":"2024-04-15T15:20:00.612258+0000","last_unstale":"2024-04-15T15:20:01.907193+0000","last_undegraded":"2024-04-15T15:20:01.907193+0000","last_fullsized":"2024-04-15T15:20:01.907193+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:53:46.359850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624975+0000","last_change":"2024-04-15T15:20:00.620494+0000","last_active":"2024-04-15T15:20:01.624975+0000","last_peered":"2024-04-15T15:20:01.624975+0000","last_clean":"2024-04-15T15:20:01.624975+0000","last_became_active":"2024-04-15T15:20:00.620229+0000","last_became_peered":"2024-04-15T15:20:00.620229+0000","last_unstale":"2024-04-15T15:20:01.624975+0000","last_undegraded":"2024-04-15T15:20:01.624975+0000","last_fullsized":"2024-04-15T15:20:01.624975+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:35:41.539419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609012+0000","last_change":"2024-04-15T15:20:00.603794+0000","last_active":"2024-04-15T15:20:01.609012+0000","last_peered":"2024-04-15T15:20:01.609012+0000","last_clean":"2024-04-15T15:20:01.609012+0000","last_became_active":"2024-04-15T15:20:00.603460+0000","last_became_peered":"2024-04-15T15:20:00.603460+0000","last_unstale":"2024-04-15T15:20:01.609012+0000","last_undegraded":"2024-04-15T15:20:01.609012+0000","last_fullsized":"2024-04-15T15:20:01.609012+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:00:29.913810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904947+0000","last_change":"2024-04-15T15:20:00.618347+0000","last_active":"2024-04-15T15:20:01.904947+0000","last_peered":"2024-04-15T15:20:01.904947+0000","last_clean":"2024-04-15T15:20:01.904947+0000","last_became_active":"2024-04-15T15:20:00.618041+0000","last_became_peered":"2024-04-15T15:20:00.618041+0000","last_unstale":"2024-04-15T15:20:01.904947+0000","last_undegraded":"2024-04-15T15:20:01.904947+0000","last_fullsized":"2024-04-15T15:20:01.904947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:05:39.404242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.606011+0000","last_change":"2024-04-15T15:20:00.611489+0000","last_active":"2024-04-15T15:20:01.606011+0000","last_peered":"2024-04-15T15:20:01.606011+0000","last_clean":"2024-04-15T15:20:01.606011+0000","last_became_active":"2024-04-15T15:20:00.607617+0000","last_became_peered":"2024-04-15T15:20:00.607617+0000","last_unstale":"2024-04-15T15:20:01.606011+0000","last_undegraded":"2024-04-15T15:20:01.606011+0000","last_fullsized":"2024-04-15T15:20:01.606011+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:30:16.127825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609663+0000","last_change":"2024-04-15T15:20:00.616963+0000","last_active":"2024-04-15T15:20:01.609663+0000","last_peered":"2024-04-15T15:20:01.609663+0000","last_clean":"2024-04-15T15:20:01.609663+0000","last_became_active":"2024-04-15T15:20:00.615614+0000","last_became_peered":"2024-04-15T15:20:00.615614+0000","last_unstale":"2024-04-15T15:20:01.609663+0000","last_undegraded":"2024-04-15T15:20:01.609663+0000","last_fullsized":"2024-04-15T15:20:01.609663+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:28:57.704150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.606878+0000","last_change":"2024-04-15T15:20:00.607968+0000","last_active":"2024-04-15T15:20:01.606878+0000","last_peered":"2024-04-15T15:20:01.606878+0000","last_clean":"2024-04-15T15:20:01.606878+0000","last_became_active":"2024-04-15T15:20:00.606951+0000","last_became_peered":"2024-04-15T15:20:00.606951+0000","last_unstale":"2024-04-15T15:20:01.606878+0000","last_undegraded":"2024-04-15T15:20:01.606878+0000","last_fullsized":"2024-04-15T15:20:01.606878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:00:29.913810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620482+0000","last_change":"2024-04-15T15:20:00.607623+0000","last_active":"2024-04-15T15:20:01.620482+0000","last_peered":"2024-04-15T15:20:01.620482+0000","last_clean":"2024-04-15T15:20:01.620482+0000","last_became_active":"2024-04-15T15:20:00.606698+0000","last_became_peered":"2024-04-15T15:20:00.606698+0000","last_unstale":"2024-04-15T15:20:01.620482+0000","last_undegraded":"2024-04-15T15:20:01.620482+0000","last_fullsized":"2024-04-15T15:20:01.620482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:20:29.604777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610534+0000","last_change":"2024-04-15T15:19:59.562229+0000","last_active":"2024-04-15T15:20:01.610534+0000","last_peered":"2024-04-15T15:20:01.610534+0000","last_clean":"2024-04-15T15:20:01.610534+0000","last_became_active":"2024-04-15T15:19:59.561906+0000","last_became_peered":"2024-04-15T15:19:59.561906+0000","last_unstale":"2024-04-15T15:20:01.610534+0000","last_undegraded":"2024-04-15T15:20:01.610534+0000","last_fullsized":"2024-04-15T15:20:01.610534+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:18.761888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610663+0000","last_change":"2024-04-15T15:20:00.614172+0000","last_active":"2024-04-15T15:20:01.610663+0000","last_peered":"2024-04-15T15:20:01.610663+0000","last_clean":"2024-04-15T15:20:01.610663+0000","last_became_active":"2024-04-15T15:20:00.613873+0000","last_became_peered":"2024-04-15T15:20:00.613873+0000","last_unstale":"2024-04-15T15:20:01.610663+0000","last_undegraded":"2024-04-15T15:20:01.610663+0000","last_fullsized":"2024-04-15T15:20:01.610663+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:20:29.604777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620480+0000","last_change":"2024-04-15T15:19:59.571252+0000","last_active":"2024-04-15T15:20:01.620480+0000","last_peered":"2024-04-15T15:20:01.620480+0000","last_clean":"2024-04-15T15:20:01.620480+0000","last_became_active":"2024-04-15T15:19:59.570943+0000","last_became_peered":"2024-04-15T15:19:59.570943+0000","last_unstale":"2024-04-15T15:20:01.620480+0000","last_undegraded":"2024-04-15T15:20:01.620480+0000","last_fullsized":"2024-04-15T15:20:01.620480+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:14:50.511195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904476+0000","last_change":"2024-04-15T15:20:00.616384+0000","last_active":"2024-04-15T15:20:01.904476+0000","last_peered":"2024-04-15T15:20:01.904476+0000","last_clean":"2024-04-15T15:20:01.904476+0000","last_became_active":"2024-04-15T15:20:00.616216+0000","last_became_peered":"2024-04-15T15:20:00.616216+0000","last_unstale":"2024-04-15T15:20:01.904476+0000","last_undegraded":"2024-04-15T15:20:01.904476+0000","last_fullsized":"2024-04-15T15:20:01.904476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:39:11.254906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906546+0000","last_change":"2024-04-15T15:19:59.558193+0000","last_active":"2024-04-15T15:20:01.906546+0000","last_peered":"2024-04-15T15:20:01.906546+0000","last_clean":"2024-04-15T15:20:01.906546+0000","last_became_active":"2024-04-15T15:19:59.557897+0000","last_became_peered":"2024-04-15T15:19:59.557897+0000","last_unstale":"2024-04-15T15:20:01.906546+0000","last_undegraded":"2024-04-15T15:20:01.906546+0000","last_fullsized":"2024-04-15T15:20:01.906546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:26:18.725082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.625003+0000","last_change":"2024-04-15T15:20:00.611926+0000","last_active":"2024-04-15T15:20:01.625003+0000","last_peered":"2024-04-15T15:20:01.625003+0000","last_clean":"2024-04-15T15:20:01.625003+0000","last_became_active":"2024-04-15T15:20:00.611607+0000","last_became_peered":"2024-04-15T15:20:00.611607+0000","last_unstale":"2024-04-15T15:20:01.625003+0000","last_undegraded":"2024-04-15T15:20:01.625003+0000","last_fullsized":"2024-04-15T15:20:01.625003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:17:36.203246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904132+0000","last_change":"2024-04-15T15:19:59.564020+0000","last_active":"2024-04-15T15:20:01.904132+0000","last_peered":"2024-04-15T15:20:01.904132+0000","last_clean":"2024-04-15T15:20:01.904132+0000","last_became_active":"2024-04-15T15:19:59.563731+0000","last_became_peered":"2024-04-15T15:19:59.563731+0000","last_unstale":"2024-04-15T15:20:01.904132+0000","last_undegraded":"2024-04-15T15:20:01.904132+0000","last_fullsized":"2024-04-15T15:20:01.904132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:41:08.033269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.607665+0000","last_change":"2024-04-15T15:20:00.612268+0000","last_active":"2024-04-15T15:20:01.607665+0000","last_peered":"2024-04-15T15:20:01.607665+0000","last_clean":"2024-04-15T15:20:01.607665+0000","last_became_active":"2024-04-15T15:20:00.611977+0000","last_became_peered":"2024-04-15T15:20:00.611977+0000","last_unstale":"2024-04-15T15:20:01.607665+0000","last_undegraded":"2024-04-15T15:20:01.607665+0000","last_fullsized":"2024-04-15T15:20:01.607665+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:02:52.890702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904701+0000","last_change":"2024-04-15T15:19:59.553449+0000","last_active":"2024-04-15T15:20:01.904701+0000","last_peered":"2024-04-15T15:20:01.904701+0000","last_clean":"2024-04-15T15:20:01.904701+0000","last_became_active":"2024-04-15T15:19:59.552881+0000","last_became_peered":"2024-04-15T15:19:59.552881+0000","last_unstale":"2024-04-15T15:20:01.904701+0000","last_undegraded":"2024-04-15T15:20:01.904701+0000","last_fullsized":"2024-04-15T15:20:01.904701+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:34:35.996734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609204+0000","last_change":"2024-04-15T15:20:00.605473+0000","last_active":"2024-04-15T15:20:01.609204+0000","last_peered":"2024-04-15T15:20:01.609204+0000","last_clean":"2024-04-15T15:20:01.609204+0000","last_became_active":"2024-04-15T15:20:00.605208+0000","last_became_peered":"2024-04-15T15:20:00.605208+0000","last_unstale":"2024-04-15T15:20:01.609204+0000","last_undegraded":"2024-04-15T15:20:01.609204+0000","last_fullsized":"2024-04-15T15:20:01.609204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:50:34.114889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904894+0000","last_change":"2024-04-15T15:19:59.549691+0000","last_active":"2024-04-15T15:20:01.904894+0000","last_peered":"2024-04-15T15:20:01.904894+0000","last_clean":"2024-04-15T15:20:01.904894+0000","last_became_active":"2024-04-15T15:19:59.549390+0000","last_became_peered":"2024-04-15T15:19:59.549390+0000","last_unstale":"2024-04-15T15:20:01.904894+0000","last_undegraded":"2024-04-15T15:20:01.904894+0000","last_fullsized":"2024-04-15T15:20:01.904894+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:11:03.397227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904149+0000","last_change":"2024-04-15T15:20:00.609329+0000","last_active":"2024-04-15T15:20:01.904149+0000","last_peered":"2024-04-15T15:20:01.904149+0000","last_clean":"2024-04-15T15:20:01.904149+0000","last_became_active":"2024-04-15T15:20:00.608999+0000","last_became_peered":"2024-04-15T15:20:00.608999+0000","last_unstale":"2024-04-15T15:20:01.904149+0000","last_undegraded":"2024-04-15T15:20:01.904149+0000","last_fullsized":"2024-04-15T15:20:01.904149+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:48:36.275786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905037+0000","last_change":"2024-04-15T15:19:59.555002+0000","last_active":"2024-04-15T15:20:01.905037+0000","last_peered":"2024-04-15T15:20:01.905037+0000","last_clean":"2024-04-15T15:20:01.905037+0000","last_became_active":"2024-04-15T15:19:59.554675+0000","last_became_peered":"2024-04-15T15:19:59.554675+0000","last_unstale":"2024-04-15T15:20:01.905037+0000","last_undegraded":"2024-04-15T15:20:01.905037+0000","last_fullsized":"2024-04-15T15:20:01.905037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:25.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624121+0000","last_change":"2024-04-15T15:20:00.616313+0000","last_active":"2024-04-15T15:20:01.624121+0000","last_peered":"2024-04-15T15:20:01.624121+0000","last_clean":"2024-04-15T15:20:01.624121+0000","last_became_active":"2024-04-15T15:20:00.614709+0000","last_became_peered":"2024-04-15T15:20:00.614709+0000","last_unstale":"2024-04-15T15:20:01.624121+0000","last_undegraded":"2024-04-15T15:20:01.624121+0000","last_fullsized":"2024-04-15T15:20:01.624121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:47:27.051203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904230+0000","last_change":"2024-04-15T15:19:59.557469+0000","last_active":"2024-04-15T15:20:01.904230+0000","last_peered":"2024-04-15T15:20:01.904230+0000","last_clean":"2024-04-15T15:20:01.904230+0000","last_became_active":"2024-04-15T15:19:59.556245+0000","last_became_peered":"2024-04-15T15:19:59.556245+0000","last_unstale":"2024-04-15T15:20:01.904230+0000","last_undegraded":"2024-04-15T15:20:01.904230+0000","last_fullsized":"2024-04-15T15:20:01.904230+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:55:45.235539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608734+0000","last_change":"2024-04-15T15:20:00.616837+0000","last_active":"2024-04-15T15:20:01.608734+0000","last_peered":"2024-04-15T15:20:01.608734+0000","last_clean":"2024-04-15T15:20:01.608734+0000","last_became_active":"2024-04-15T15:20:00.615232+0000","last_became_peered":"2024-04-15T15:20:00.615232+0000","last_unstale":"2024-04-15T15:20:01.608734+0000","last_undegraded":"2024-04-15T15:20:01.608734+0000","last_fullsized":"2024-04-15T15:20:01.608734+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:02:52.890702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906891+0000","last_change":"2024-04-15T15:19:59.566367+0000","last_active":"2024-04-15T15:20:01.906891+0000","last_peered":"2024-04-15T15:20:01.906891+0000","last_clean":"2024-04-15T15:20:01.906891+0000","last_became_active":"2024-04-15T15:19:59.566080+0000","last_became_peered":"2024-04-15T15:19:59.566080+0000","last_unstale":"2024-04-15T15:20:01.906891+0000","last_undegraded":"2024-04-15T15:20:01.906891+0000","last_fullsized":"2024-04-15T15:20:01.906891+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:55:13.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906949+0000","last_change":"2024-04-15T15:20:00.621744+0000","last_active":"2024-04-15T15:20:01.906949+0000","last_peered":"2024-04-15T15:20:01.906949+0000","last_clean":"2024-04-15T15:20:01.906949+0000","last_became_active":"2024-04-15T15:20:00.621464+0000","last_became_peered":"2024-04-15T15:20:00.621464+0000","last_unstale":"2024-04-15T15:20:01.906949+0000","last_undegraded":"2024-04-15T15:20:01.906949+0000","last_fullsized":"2024-04-15T15:20:01.906949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:44:14.986046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620602+0000","last_change":"2024-04-15T15:19:59.557592+0000","last_active":"2024-04-15T15:20:01.620602+0000","last_peered":"2024-04-15T15:20:01.620602+0000","last_clean":"2024-04-15T15:20:01.620602+0000","last_became_active":"2024-04-15T15:19:59.557267+0000","last_became_peered":"2024-04-15T15:19:59.557267+0000","last_unstale":"2024-04-15T15:20:01.620602+0000","last_undegraded":"2024-04-15T15:20:01.620602+0000","last_fullsized":"2024-04-15T15:20:01.620602+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:07:49.627900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906127+0000","last_change":"2024-04-15T15:20:00.608598+0000","last_active":"2024-04-15T15:20:01.906127+0000","last_peered":"2024-04-15T15:20:01.906127+0000","last_clean":"2024-04-15T15:20:01.906127+0000","last_became_active":"2024-04-15T15:20:00.608275+0000","last_became_peered":"2024-04-15T15:20:00.608275+0000","last_unstale":"2024-04-15T15:20:01.906127+0000","last_undegraded":"2024-04-15T15:20:01.906127+0000","last_fullsized":"2024-04-15T15:20:01.906127+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:36:47.757839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608437+0000","last_change":"2024-04-15T15:19:59.566469+0000","last_active":"2024-04-15T15:20:01.608437+0000","last_peered":"2024-04-15T15:20:01.608437+0000","last_clean":"2024-04-15T15:20:01.608437+0000","last_became_active":"2024-04-15T15:19:59.566154+0000","last_became_peered":"2024-04-15T15:19:59.566154+0000","last_unstale":"2024-04-15T15:20:01.608437+0000","last_undegraded":"2024-04-15T15:20:01.608437+0000","last_fullsized":"2024-04-15T15:20:01.608437+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:28:56.698937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.608101+0000","last_change":"2024-04-15T15:20:00.611802+0000","last_active":"2024-04-15T15:20:01.608101+0000","last_peered":"2024-04-15T15:20:01.608101+0000","last_clean":"2024-04-15T15:20:01.608101+0000","last_became_active":"2024-04-15T15:20:00.611108+0000","last_became_peered":"2024-04-15T15:20:00.611108+0000","last_unstale":"2024-04-15T15:20:01.608101+0000","last_undegraded":"2024-04-15T15:20:01.608101+0000","last_fullsized":"2024-04-15T15:20:01.608101+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:01:26.510445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906227+0000","last_change":"2024-04-15T15:19:59.559002+0000","last_active":"2024-04-15T15:20:01.906227+0000","last_peered":"2024-04-15T15:20:01.906227+0000","last_clean":"2024-04-15T15:20:01.906227+0000","last_became_active":"2024-04-15T15:19:59.558721+0000","last_became_peered":"2024-04-15T15:19:59.558721+0000","last_unstale":"2024-04-15T15:20:01.906227+0000","last_undegraded":"2024-04-15T15:20:01.906227+0000","last_fullsized":"2024-04-15T15:20:01.906227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:20:28.599563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624263+0000","last_change":"2024-04-15T15:20:00.610191+0000","last_active":"2024-04-15T15:20:01.624263+0000","last_peered":"2024-04-15T15:20:01.624263+0000","last_clean":"2024-04-15T15:20:01.624263+0000","last_became_active":"2024-04-15T15:20:00.609856+0000","last_became_peered":"2024-04-15T15:20:00.609856+0000","last_unstale":"2024-04-15T15:20:01.624263+0000","last_undegraded":"2024-04-15T15:20:01.624263+0000","last_fullsized":"2024-04-15T15:20:01.624263+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:04:28.710555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905009+0000","last_change":"2024-04-15T15:19:59.558430+0000","last_active":"2024-04-15T15:20:01.905009+0000","last_peered":"2024-04-15T15:20:01.905009+0000","last_clean":"2024-04-15T15:20:01.905009+0000","last_became_active":"2024-04-15T15:19:59.557976+0000","last_became_peered":"2024-04-15T15:19:59.557976+0000","last_unstale":"2024-04-15T15:20:01.905009+0000","last_undegraded":"2024-04-15T15:20:01.905009+0000","last_fullsized":"2024-04-15T15:20:01.905009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:46:18.671155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.906118+0000","last_change":"2024-04-15T15:20:00.617234+0000","last_active":"2024-04-15T15:20:01.906118+0000","last_peered":"2024-04-15T15:20:01.906118+0000","last_clean":"2024-04-15T15:20:01.906118+0000","last_became_active":"2024-04-15T15:20:00.616888+0000","last_became_peered":"2024-04-15T15:20:00.616888+0000","last_unstale":"2024-04-15T15:20:01.906118+0000","last_undegraded":"2024-04-15T15:20:01.906118+0000","last_fullsized":"2024-04-15T15:20:01.906118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:50:34.114889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624403+0000","last_change":"2024-04-15T15:19:59.556664+0000","last_active":"2024-04-15T15:20:01.624403+0000","last_peered":"2024-04-15T15:20:01.624403+0000","last_clean":"2024-04-15T15:20:01.624403+0000","last_became_active":"2024-04-15T15:19:59.556350+0000","last_became_peered":"2024-04-15T15:19:59.556350+0000","last_unstale":"2024-04-15T15:20:01.624403+0000","last_undegraded":"2024-04-15T15:20:01.624403+0000","last_fullsized":"2024-04-15T15:20:01.624403+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:14:47.709963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.605395+0000","last_change":"2024-04-15T15:20:00.612658+0000","last_active":"2024-04-15T15:20:01.605395+0000","last_peered":"2024-04-15T15:20:01.605395+0000","last_clean":"2024-04-15T15:20:01.605395+0000","last_became_active":"2024-04-15T15:20:00.612422+0000","last_became_peered":"2024-04-15T15:20:00.612422+0000","last_unstale":"2024-04-15T15:20:01.605395+0000","last_undegraded":"2024-04-15T15:20:01.605395+0000","last_fullsized":"2024-04-15T15:20:01.605395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:38:12.140232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905149+0000","last_change":"2024-04-15T15:19:59.567169+0000","last_active":"2024-04-15T15:20:01.905149+0000","last_peered":"2024-04-15T15:20:01.905149+0000","last_clean":"2024-04-15T15:20:01.905149+0000","last_became_active":"2024-04-15T15:19:59.566906+0000","last_became_peered":"2024-04-15T15:19:59.566906+0000","last_unstale":"2024-04-15T15:20:01.905149+0000","last_undegraded":"2024-04-15T15:20:01.905149+0000","last_fullsized":"2024-04-15T15:20:01.905149+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:51:03.017053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609691+0000","last_change":"2024-04-15T15:20:00.613749+0000","last_active":"2024-04-15T15:20:01.609691+0000","last_peered":"2024-04-15T15:20:01.609691+0000","last_clean":"2024-04-15T15:20:01.609691+0000","last_became_active":"2024-04-15T15:20:00.613409+0000","last_became_peered":"2024-04-15T15:20:00.613409+0000","last_unstale":"2024-04-15T15:20:01.609691+0000","last_undegraded":"2024-04-15T15:20:01.609691+0000","last_fullsized":"2024-04-15T15:20:01.609691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:26.988652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.905270+0000","last_change":"2024-04-15T15:19:59.558986+0000","last_active":"2024-04-15T15:20:01.905270+0000","last_peered":"2024-04-15T15:20:01.905270+0000","last_clean":"2024-04-15T15:20:01.905270+0000","last_became_active":"2024-04-15T15:19:59.557861+0000","last_became_peered":"2024-04-15T15:19:59.557861+0000","last_unstale":"2024-04-15T15:20:01.905270+0000","last_undegraded":"2024-04-15T15:20:01.905270+0000","last_fullsized":"2024-04-15T15:20:01.905270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:28:56.698937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904955+0000","last_change":"2024-04-15T15:20:00.602513+0000","last_active":"2024-04-15T15:20:01.904955+0000","last_peered":"2024-04-15T15:20:01.904955+0000","last_clean":"2024-04-15T15:20:01.904955+0000","last_became_active":"2024-04-15T15:20:00.601487+0000","last_became_peered":"2024-04-15T15:20:00.601487+0000","last_unstale":"2024-04-15T15:20:01.904955+0000","last_undegraded":"2024-04-15T15:20:01.904955+0000","last_fullsized":"2024-04-15T15:20:01.904955+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:50:35.288023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.625024+0000","last_change":"2024-04-15T15:19:59.569146+0000","last_active":"2024-04-15T15:20:01.625024+0000","last_peered":"2024-04-15T15:20:01.625024+0000","last_clean":"2024-04-15T15:20:01.625024+0000","last_became_active":"2024-04-15T15:19:59.567258+0000","last_became_peered":"2024-04-15T15:19:59.567258+0000","last_unstale":"2024-04-15T15:20:01.625024+0000","last_undegraded":"2024-04-15T15:20:01.625024+0000","last_fullsized":"2024-04-15T15:20:01.625024+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:07:49.627900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609754+0000","last_change":"2024-04-15T15:20:00.597695+0000","last_active":"2024-04-15T15:20:01.609754+0000","last_peered":"2024-04-15T15:20:01.609754+0000","last_clean":"2024-04-15T15:20:01.609754+0000","last_became_active":"2024-04-15T15:20:00.597397+0000","last_became_peered":"2024-04-15T15:20:00.597397+0000","last_unstale":"2024-04-15T15:20:01.609754+0000","last_undegraded":"2024-04-15T15:20:01.609754+0000","last_fullsized":"2024-04-15T15:20:01.609754+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:36:47.757839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904950+0000","last_change":"2024-04-15T15:19:59.558312+0000","last_active":"2024-04-15T15:20:01.904950+0000","last_peered":"2024-04-15T15:20:01.904950+0000","last_clean":"2024-04-15T15:20:01.904950+0000","last_became_active":"2024-04-15T15:19:59.557829+0000","last_became_peered":"2024-04-15T15:19:59.557829+0000","last_unstale":"2024-04-15T15:20:01.904950+0000","last_undegraded":"2024-04-15T15:20:01.904950+0000","last_fullsized":"2024-04-15T15:20:01.904950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:46:37.671572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624442+0000","last_change":"2024-04-15T15:20:00.611135+0000","last_active":"2024-04-15T15:20:01.624442+0000","last_peered":"2024-04-15T15:20:01.624442+0000","last_clean":"2024-04-15T15:20:01.624442+0000","last_became_active":"2024-04-15T15:20:00.610872+0000","last_became_peered":"2024-04-15T15:20:00.610872+0000","last_unstale":"2024-04-15T15:20:01.624442+0000","last_undegraded":"2024-04-15T15:20:01.624442+0000","last_fullsized":"2024-04-15T15:20:01.624442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:01:26.510445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904901+0000","last_change":"2024-04-15T15:19:59.564696+0000","last_active":"2024-04-15T15:20:01.904901+0000","last_peered":"2024-04-15T15:20:01.904901+0000","last_clean":"2024-04-15T15:20:01.904901+0000","last_became_active":"2024-04-15T15:19:59.562001+0000","last_became_peered":"2024-04-15T15:19:59.562001+0000","last_unstale":"2024-04-15T15:20:01.904901+0000","last_undegraded":"2024-04-15T15:20:01.904901+0000","last_fullsized":"2024-04-15T15:20:01.904901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:06:05.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904272+0000","last_change":"2024-04-15T15:20:00.624563+0000","last_active":"2024-04-15T15:20:01.904272+0000","last_peered":"2024-04-15T15:20:01.904272+0000","last_clean":"2024-04-15T15:20:01.904272+0000","last_became_active":"2024-04-15T15:20:00.624235+0000","last_became_peered":"2024-04-15T15:20:00.624235+0000","last_unstale":"2024-04-15T15:20:01.904272+0000","last_undegraded":"2024-04-15T15:20:01.904272+0000","last_fullsized":"2024-04-15T15:20:01.904272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:57:21.317994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904354+0000","last_change":"2024-04-15T15:19:59.560819+0000","last_active":"2024-04-15T15:20:01.904354+0000","last_peered":"2024-04-15T15:20:01.904354+0000","last_clean":"2024-04-15T15:20:01.904354+0000","last_became_active":"2024-04-15T15:19:59.560471+0000","last_became_peered":"2024-04-15T15:19:59.560471+0000","last_unstale":"2024-04-15T15:20:01.904354+0000","last_undegraded":"2024-04-15T15:20:01.904354+0000","last_fullsized":"2024-04-15T15:20:01.904354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:58:32.114685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903548+0000","last_change":"2024-04-15T15:20:00.626744+0000","last_active":"2024-04-15T15:20:01.903548+0000","last_peered":"2024-04-15T15:20:01.903548+0000","last_clean":"2024-04-15T15:20:01.903548+0000","last_became_active":"2024-04-15T15:20:00.621491+0000","last_became_peered":"2024-04-15T15:20:00.621491+0000","last_unstale":"2024-04-15T15:20:01.903548+0000","last_undegraded":"2024-04-15T15:20:01.903548+0000","last_fullsized":"2024-04-15T15:20:01.903548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:53:54.386402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620531+0000","last_change":"2024-04-15T15:19:59.568281+0000","last_active":"2024-04-15T15:20:01.620531+0000","last_peered":"2024-04-15T15:20:01.620531+0000","last_clean":"2024-04-15T15:20:01.620531+0000","last_became_active":"2024-04-15T15:19:59.567805+0000","last_became_peered":"2024-04-15T15:19:59.567805+0000","last_unstale":"2024-04-15T15:20:01.620531+0000","last_undegraded":"2024-04-15T15:20:01.620531+0000","last_fullsized":"2024-04-15T15:20:01.620531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:57:20.312781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903601+0000","last_change":"2024-04-15T15:20:00.628107+0000","last_active":"2024-04-15T15:20:01.903601+0000","last_peered":"2024-04-15T15:20:01.903601+0000","last_clean":"2024-04-15T15:20:01.903601+0000","last_became_active":"2024-04-15T15:20:00.627798+0000","last_became_peered":"2024-04-15T15:20:00.627798+0000","last_unstale":"2024-04-15T15:20:01.903601+0000","last_undegraded":"2024-04-15T15:20:01.903601+0000","last_fullsized":"2024-04-15T15:20:01.903601+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:19.767102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610052+0000","last_change":"2024-04-15T15:19:59.564372+0000","last_active":"2024-04-15T15:20:01.610052+0000","last_peered":"2024-04-15T15:20:01.610052+0000","last_clean":"2024-04-15T15:20:01.610052+0000","last_became_active":"2024-04-15T15:19:59.564094+0000","last_became_peered":"2024-04-15T15:19:59.564094+0000","last_unstale":"2024-04-15T15:20:01.610052+0000","last_undegraded":"2024-04-15T15:20:01.610052+0000","last_fullsized":"2024-04-15T15:20:01.610052+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:26:21.787578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903392+0000","last_change":"2024-04-15T15:20:00.626889+0000","last_active":"2024-04-15T15:20:01.903392+0000","last_peered":"2024-04-15T15:20:01.903392+0000","last_clean":"2024-04-15T15:20:01.903392+0000","last_became_active":"2024-04-15T15:20:00.626572+0000","last_became_peered":"2024-04-15T15:20:00.626572+0000","last_unstale":"2024-04-15T15:20:01.903392+0000","last_undegraded":"2024-04-15T15:20:01.903392+0000","last_fullsized":"2024-04-15T15:20:01.903392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:44:47.893636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903969+0000","last_change":"2024-04-15T15:19:59.565493+0000","last_active":"2024-04-15T15:20:01.903969+0000","last_peered":"2024-04-15T15:20:01.903969+0000","last_clean":"2024-04-15T15:20:01.903969+0000","last_became_active":"2024-04-15T15:19:59.565208+0000","last_became_peered":"2024-04-15T15:19:59.565208+0000","last_unstale":"2024-04-15T15:20:01.903969+0000","last_undegraded":"2024-04-15T15:20:01.903969+0000","last_fullsized":"2024-04-15T15:20:01.903969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:49:52.079334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.605031+0000","last_change":"2024-04-15T15:20:00.609949+0000","last_active":"2024-04-15T15:20:01.605031+0000","last_peered":"2024-04-15T15:20:01.605031+0000","last_clean":"2024-04-15T15:20:01.605031+0000","last_became_active":"2024-04-15T15:20:00.609511+0000","last_became_peered":"2024-04-15T15:20:00.609511+0000","last_unstale":"2024-04-15T15:20:01.605031+0000","last_undegraded":"2024-04-15T15:20:01.605031+0000","last_fullsized":"2024-04-15T15:20:01.605031+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:00:21.659268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.607589+0000","last_change":"2024-04-15T15:19:59.566727+0000","last_active":"2024-04-15T15:20:01.607589+0000","last_peered":"2024-04-15T15:20:01.607589+0000","last_clean":"2024-04-15T15:20:01.607589+0000","last_became_active":"2024-04-15T15:19:59.566169+0000","last_became_peered":"2024-04-15T15:19:59.566169+0000","last_unstale":"2024-04-15T15:20:01.607589+0000","last_undegraded":"2024-04-15T15:20:01.607589+0000","last_fullsized":"2024-04-15T15:20:01.607589+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:25.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904431+0000","last_change":"2024-04-15T15:20:00.624647+0000","last_active":"2024-04-15T15:20:01.904431+0000","last_peered":"2024-04-15T15:20:01.904431+0000","last_clean":"2024-04-15T15:20:01.904431+0000","last_became_active":"2024-04-15T15:20:00.624327+0000","last_became_peered":"2024-04-15T15:20:00.624327+0000","last_unstale":"2024-04-15T15:20:01.904431+0000","last_undegraded":"2024-04-15T15:20:01.904431+0000","last_fullsized":"2024-04-15T15:20:01.904431+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:14:51.516409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.625034+0000","last_change":"2024-04-15T15:19:59.570791+0000","last_active":"2024-04-15T15:20:01.625034+0000","last_peered":"2024-04-15T15:20:01.625034+0000","last_clean":"2024-04-15T15:20:01.625034+0000","last_became_active":"2024-04-15T15:19:59.570453+0000","last_became_peered":"2024-04-15T15:19:59.570453+0000","last_unstale":"2024-04-15T15:20:01.625034+0000","last_undegraded":"2024-04-15T15:20:01.625034+0000","last_fullsized":"2024-04-15T15:20:01.625034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:14:42.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610556+0000","last_change":"2024-04-15T15:20:00.614557+0000","last_active":"2024-04-15T15:20:01.610556+0000","last_peered":"2024-04-15T15:20:01.610556+0000","last_clean":"2024-04-15T15:20:01.610556+0000","last_became_active":"2024-04-15T15:20:00.614289+0000","last_became_peered":"2024-04-15T15:20:00.614289+0000","last_unstale":"2024-04-15T15:20:01.610556+0000","last_undegraded":"2024-04-15T15:20:01.610556+0000","last_fullsized":"2024-04-15T15:20:01.610556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:06:53.187316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.623360+0000","last_change":"2024-04-15T15:19:59.565639+0000","last_active":"2024-04-15T15:20:01.623360+0000","last_peered":"2024-04-15T15:20:01.623360+0000","last_clean":"2024-04-15T15:20:01.623360+0000","last_became_active":"2024-04-15T15:19:59.565287+0000","last_became_peered":"2024-04-15T15:19:59.565287+0000","last_unstale":"2024-04-15T15:20:01.623360+0000","last_undegraded":"2024-04-15T15:20:01.623360+0000","last_fullsized":"2024-04-15T15:20:01.623360+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:14:42.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620514+0000","last_change":"2024-04-15T15:20:00.605815+0000","last_active":"2024-04-15T15:20:01.620514+0000","last_peered":"2024-04-15T15:20:01.620514+0000","last_clean":"2024-04-15T15:20:01.620514+0000","last_became_active":"2024-04-15T15:20:00.605350+0000","last_became_peered":"2024-04-15T15:20:00.605350+0000","last_unstale":"2024-04-15T15:20:01.620514+0000","last_undegraded":"2024-04-15T15:20:01.620514+0000","last_fullsized":"2024-04-15T15:20:01.620514+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:26:19.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624667+0000","last_change":"2024-04-15T15:19:59.560517+0000","last_active":"2024-04-15T15:20:01.624667+0000","last_peered":"2024-04-15T15:20:01.624667+0000","last_clean":"2024-04-15T15:20:01.624667+0000","last_became_active":"2024-04-15T15:19:59.560021+0000","last_became_peered":"2024-04-15T15:19:59.560021+0000","last_unstale":"2024-04-15T15:20:01.624667+0000","last_undegraded":"2024-04-15T15:20:01.624667+0000","last_fullsized":"2024-04-15T15:20:01.624667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:14:50.511195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.609669+0000","last_change":"2024-04-15T15:20:00.619344+0000","last_active":"2024-04-15T15:20:01.609669+0000","last_peered":"2024-04-15T15:20:01.609669+0000","last_clean":"2024-04-15T15:20:01.609669+0000","last_became_active":"2024-04-15T15:20:00.619057+0000","last_became_peered":"2024-04-15T15:20:00.619057+0000","last_unstale":"2024-04-15T15:20:01.609669+0000","last_undegraded":"2024-04-15T15:20:01.609669+0000","last_fullsized":"2024-04-15T15:20:01.609669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:55:31.770386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620710+0000","last_change":"2024-04-15T15:19:59.560841+0000","last_active":"2024-04-15T15:20:01.620710+0000","last_peered":"2024-04-15T15:20:01.620710+0000","last_clean":"2024-04-15T15:20:01.620710+0000","last_became_active":"2024-04-15T15:19:59.560515+0000","last_became_peered":"2024-04-15T15:19:59.560515+0000","last_unstale":"2024-04-15T15:20:01.620710+0000","last_undegraded":"2024-04-15T15:20:01.620710+0000","last_fullsized":"2024-04-15T15:20:01.620710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:14:47.709963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.620585+0000","last_change":"2024-04-15T15:20:00.613259+0000","last_active":"2024-04-15T15:20:01.620585+0000","last_peered":"2024-04-15T15:20:01.620585+0000","last_clean":"2024-04-15T15:20:01.620585+0000","last_became_active":"2024-04-15T15:20:00.612923+0000","last_became_peered":"2024-04-15T15:20:00.612923+0000","last_unstale":"2024-04-15T15:20:01.620585+0000","last_undegraded":"2024-04-15T15:20:01.620585+0000","last_fullsized":"2024-04-15T15:20:01.620585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:55:14.237241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.624544+0000","last_change":"2024-04-15T15:19:59.569507+0000","last_active":"2024-04-15T15:20:01.624544+0000","last_peered":"2024-04-15T15:20:01.624544+0000","last_clean":"2024-04-15T15:20:01.624544+0000","last_became_active":"2024-04-15T15:19:59.568995+0000","last_became_peered":"2024-04-15T15:19:59.568995+0000","last_unstale":"2024-04-15T15:20:01.624544+0000","last_undegraded":"2024-04-15T15:20:01.624544+0000","last_fullsized":"2024-04-15T15:20:01.624544+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:32:06.507059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904841+0000","last_change":"2024-04-15T15:20:00.612377+0000","last_active":"2024-04-15T15:20:01.904841+0000","last_peered":"2024-04-15T15:20:01.904841+0000","last_clean":"2024-04-15T15:20:01.904841+0000","last_became_active":"2024-04-15T15:20:00.612064+0000","last_became_peered":"2024-04-15T15:20:00.612064+0000","last_unstale":"2024-04-15T15:20:01.904841+0000","last_undegraded":"2024-04-15T15:20:01.904841+0000","last_fullsized":"2024-04-15T15:20:01.904841+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:17:36.203246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904268+0000","last_change":"2024-04-15T15:19:59.562470+0000","last_active":"2024-04-15T15:20:01.904268+0000","last_peered":"2024-04-15T15:20:01.904268+0000","last_clean":"2024-04-15T15:20:01.904268+0000","last_became_active":"2024-04-15T15:19:59.560405+0000","last_became_peered":"2024-04-15T15:19:59.560405+0000","last_unstale":"2024-04-15T15:20:01.904268+0000","last_undegraded":"2024-04-15T15:20:01.904268+0000","last_fullsized":"2024-04-15T15:20:01.904268+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:28:01.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.903985+0000","last_change":"2024-04-15T15:20:00.615924+0000","last_active":"2024-04-15T15:20:01.903985+0000","last_peered":"2024-04-15T15:20:01.903985+0000","last_clean":"2024-04-15T15:20:01.903985+0000","last_became_active":"2024-04-15T15:20:00.615644+0000","last_became_peered":"2024-04-15T15:20:00.615644+0000","last_unstale":"2024-04-15T15:20:01.903985+0000","last_undegraded":"2024-04-15T15:20:01.903985+0000","last_fullsized":"2024-04-15T15:20:01.903985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:44:57.445057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.904978+0000","last_change":"2024-04-15T15:19:59.570049+0000","last_active":"2024-04-15T15:20:01.904978+0000","last_peered":"2024-04-15T15:20:01.904978+0000","last_clean":"2024-04-15T15:20:01.904978+0000","last_became_active":"2024-04-15T15:19:59.568744+0000","last_became_peered":"2024-04-15T15:19:59.568744+0000","last_unstale":"2024-04-15T15:20:01.904978+0000","last_undegraded":"2024-04-15T15:20:01.904978+0000","last_fullsized":"2024-04-15T15:20:01.904978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:55:30.765172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.607013+0000","last_change":"2024-04-15T15:20:00.624541+0000","last_active":"2024-04-15T15:20:01.607013+0000","last_peered":"2024-04-15T15:20:01.607013+0000","last_clean":"2024-04-15T15:20:01.607013+0000","last_became_active":"2024-04-15T15:20:00.624291+0000","last_became_peered":"2024-04-15T15:20:00.624291+0000","last_unstale":"2024-04-15T15:20:01.607013+0000","last_undegraded":"2024-04-15T15:20:01.607013+0000","last_fullsized":"2024-04-15T15:20:01.607013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:59.519265+0000","last_clean_scrub_stamp":"2024-04-15T15:19:59.519265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:50:34.114889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.610154+0000","last_change":"2024-04-15T15:19:59.563726+0000","last_active":"2024-04-15T15:20:01.610154+0000","last_peered":"2024-04-15T15:20:01.610154+0000","last_clean":"2024-04-15T15:20:01.610154+0000","last_became_active":"2024-04-15T15:19:59.563395+0000","last_became_peered":"2024-04-15T15:19:59.563395+0000","last_unstale":"2024-04-15T15:20:01.610154+0000","last_undegraded":"2024-04-15T15:20:01.610154+0000","last_fullsized":"2024-04-15T15:20:01.610154+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:19:58.514051+0000","last_clean_scrub_stamp":"2024-04-15T15:19:58.514051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:43:38.542073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:20:01.606369+0000","last_change":"2024-04-15T15:18:41.162887+0000","last_active":"2024-04-15T15:20:01.606369+0000","last_peered":"2024-04-15T15:20:01.606369+0000","last_clean":"2024-04-15T15:20:01.606369+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:20:01.606369+0000","last_undegraded":"2024-04-15T15:20:01.606369+0000","last_fullsized":"2024-04-15T15:20:01.606369+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_clean_scrub_stamp":"2024-04-15T15:18:39.644354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:32:00.047414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":991232,"data_stored":745040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":155198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":155254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":409600,"data_stored":156216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":385024,"data_stored":148352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":409600,"data_stored":156216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":744950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:20:05.650 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-15T15:20:05.650 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-15T15:20:05.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.924+0000 7f707e400640 1 Processor -- start 2024-04-15T15:20:05.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.925+0000 7f707e400640 1 -- start start 2024-04-15T15:20:05.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.927+0000 7f707e400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f70780b2c30 0x7f70780728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.927+0000 7f707e400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f707806e660 con 0x7f707806f420 2024-04-15T15:20:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.927+0000 7f707e400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f707806e940 con 0x7f7078146240 2024-04-15T15:20:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.927+0000 7f707e400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7078146620 con 0x7f70780b2c30 2024-04-15T15:20:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.927+0000 7f706fe00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f707806f420 0x7f70780721a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:40400/0 (socket says 172.21.15.49:40400) 2024-04-15T15:20:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.927+0000 7f706fe00640 1 -- 172.21.15.49:0/544299390 learned_addr learned my addr 172.21.15.49:0/544299390 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:05.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.927+0000 7f707da00640 1 --2- 172.21.15.49:0/544299390 >> v2:172.21.15.49:3300/0 conn(0x7f70780b2c30 0x7f70780728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.928+0000 7f7077e00640 1 -- 172.21.15.49:0/544299390 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1139168679 0 0) 0x7f707806e660 con 0x7f707806f420 2024-04-15T15:20:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.928+0000 7f7077e00640 1 -- 172.21.15.49:0/544299390 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f705c003520 con 0x7f707806f420 2024-04-15T15:20:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.928+0000 7f707da00640 1 -- 172.21.15.49:0/544299390 >> v1:172.21.15.107:6789/0 conn(0x7f7078146240 legacy=0x7f70780b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.928+0000 7f707da00640 1 -- 172.21.15.49:0/544299390 >> v1:172.21.15.49:6789/0 conn(0x7f707806f420 legacy=0x7f70780721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.928+0000 7f707da00640 1 -- 172.21.15.49:0/544299390 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707806ec20 con 0x7f70780b2c30 2024-04-15T15:20:05.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.929+0000 7f707da00640 1 --2- 172.21.15.49:0/544299390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70780b2c30 0x7f70780728b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7070009d90 tx=0x7f707002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bcd88b93675d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.929+0000 7f7077e00640 1 -- 172.21.15.49:0/544299390 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070039070 con 0x7f70780b2c30 2024-04-15T15:20:05.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.930+0000 7f7077e00640 1 -- 172.21.15.49:0/544299390 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707002cd00 con 0x7f70780b2c30 2024-04-15T15:20:05.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.930+0000 7f7077e00640 1 -- 172.21.15.49:0/544299390 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070034640 con 0x7f70780b2c30 2024-04-15T15:20:05.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.930+0000 7f707e400640 1 -- 172.21.15.49:0/544299390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70780b2c30 msgr2=0x7f70780728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.930+0000 7f707e400640 1 --2- 172.21.15.49:0/544299390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70780b2c30 0x7f70780728b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7070009d90 tx=0x7f707002bf50 comp rx=0 tx=0).stop 2024-04-15T15:20:05.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.930+0000 7f707e400640 1 -- 172.21.15.49:0/544299390 shutdown_connections 2024-04-15T15:20:05.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.930+0000 7f707e400640 1 --2- 172.21.15.49:0/544299390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70780b2c30 0x7f70780728b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.930+0000 7f707e400640 1 -- 172.21.15.49:0/544299390 >> 172.21.15.49:0/544299390 conn(0x7f707806b930 msgr2=0x7f707806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:05.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.930+0000 7f707e400640 1 -- 172.21.15.49:0/544299390 shutdown_connections 2024-04-15T15:20:05.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.930+0000 7f707e400640 1 -- 172.21.15.49:0/544299390 wait complete. 2024-04-15T15:20:05.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.932+0000 7f707e400640 1 Processor -- start 2024-04-15T15:20:05.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.932+0000 7f707e400640 1 -- start start 2024-04-15T15:20:05.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.933+0000 7f707e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707806f420 0x7f707815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.933+0000 7f707e400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70780b2c30 0x7f70780af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.933+0000 7f707e400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7078146240 0x7f70781442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.933+0000 7f707e400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7078167330 con 0x7f7078146240 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.933+0000 7f707e400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f707815dd20 con 0x7f707806f420 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.933+0000 7f707e400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f70781674b0 con 0x7f70780b2c30 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707da00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7078146240 0x7f70781442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707da00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7078146240 0x7f70781442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42868/0 (socket says 172.21.15.49:42868) 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707da00640 1 -- 172.21.15.49:0/3473404937 learned_addr learned my addr 172.21.15.49:0/3473404937 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707d000640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707806f420 0x7f707815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f706fe00640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70780b2c30 0x7f70780af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707da00640 1 -- 172.21.15.49:0/3473404937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70780b2c30 msgr2=0x7f70780af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707da00640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70780b2c30 0x7f70780af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707da00640 1 -- 172.21.15.49:0/3473404937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707806f420 msgr2=0x7f707815c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:05.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707da00640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707806f420 0x7f707815c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:05.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707da00640 1 -- 172.21.15.49:0/3473404937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707815c750 con 0x7f7078146240 2024-04-15T15:20:05.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707d000640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707806f420 0x7f707815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:05.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.934+0000 7f707da00640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7078146240 0x7f70781442b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7070009d90 tx=0x7f7070004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.935+0000 7f7075600640 1 -- 172.21.15.49:0/3473404937 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070042020 con 0x7f7078146240 2024-04-15T15:20:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.935+0000 7f7075600640 1 -- 172.21.15.49:0/3473404937 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707003a040 con 0x7f7078146240 2024-04-15T15:20:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.935+0000 7f707e400640 1 -- 172.21.15.49:0/3473404937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7078140b80 con 0x7f7078146240 2024-04-15T15:20:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.935+0000 7f7075600640 1 -- 172.21.15.49:0/3473404937 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070039070 con 0x7f7078146240 2024-04-15T15:20:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.935+0000 7f707e400640 1 -- 172.21.15.49:0/3473404937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7078141420 con 0x7f7078146240 2024-04-15T15:20:05.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.936+0000 7f707e400640 1 -- 172.21.15.49:0/3473404937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70780755c0 con 0x7f7078146240 2024-04-15T15:20:05.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.937+0000 7f7075600640 1 -- 172.21.15.49:0/3473404937 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7070005d10 con 0x7f7078146240 2024-04-15T15:20:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.938+0000 7f7075600640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f705406c940 0x7f705406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.938+0000 7f7075600640 1 -- 172.21.15.49:0/3473404937 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f70700add40 con 0x7f7078146240 2024-04-15T15:20:05.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.939+0000 7f707d000640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f705406c940 0x7f705406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:05.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.939+0000 7f707d000640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f705406c940 0x7f705406ee00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7068000f60 tx=0x7f7068009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:05.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:05.943+0000 7f7075600640 1 -- 172.21.15.49:0/3473404937 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f707004c070 con 0x7f7078146240 2024-04-15T15:20:06.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.229+0000 7f707e400640 1 -- 172.21.15.49:0/3473404937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7078144050 con 0x7f7078146240 2024-04-15T15:20:06.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.233+0000 7f7075600640 1 -- 172.21.15.49:0/3473404937 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f707007c310 con 0x7f7078146240 2024-04-15T15:20:06.234 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-04-15T15:20:06.235 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAWRR1mkYQSHRAA0LcNsz2EApVHL/mhDGmMVg== 2024-04-15T15:20:06.235 INFO:teuthology.orchestra.run.smithi049.stdout: caps mds = "allow" 2024-04-15T15:20:06.235 INFO:teuthology.orchestra.run.smithi049.stdout: caps mon = "allow r" 2024-04-15T15:20:06.235 INFO:teuthology.orchestra.run.smithi049.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-15T15:20:06.235 INFO:teuthology.orchestra.run.smithi049.stderr:updated caps for client.0 2024-04-15T15:20:06.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.236+0000 7f707e400640 1 -- 172.21.15.49:0/3473404937 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f705406c940 msgr2=0x7f705406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.236+0000 7f707e400640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f705406c940 0x7f705406ee00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7068000f60 tx=0x7f7068009040 comp rx=0 tx=0).stop 2024-04-15T15:20:06.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.236+0000 7f707e400640 1 -- 172.21.15.49:0/3473404937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7078146240 msgr2=0x7f70781442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.236+0000 7f707e400640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7078146240 0x7f70781442b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7070009d90 tx=0x7f7070004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:06.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.236+0000 7f707e400640 1 -- 172.21.15.49:0/3473404937 shutdown_connections 2024-04-15T15:20:06.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.236+0000 7f707e400640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f705406c940 0x7f705406ee00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.236+0000 7f707e400640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70780b2c30 0x7f70780af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.236+0000 7f707e400640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7078146240 0x7f70781442b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.236+0000 7f707e400640 1 --2- 172.21.15.49:0/3473404937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707806f420 0x7f707815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.236+0000 7f707e400640 1 -- 172.21.15.49:0/3473404937 >> 172.21.15.49:0/3473404937 conn(0x7f707806b930 msgr2=0x7f707815efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:06.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.237+0000 7f707e400640 1 -- 172.21.15.49:0/3473404937 shutdown_connections 2024-04-15T15:20:06.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.237+0000 7f707e400640 1 -- 172.21.15.49:0/3473404937 wait complete. 2024-04-15T15:20:06.259 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:06.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.487+0000 7fa07be00640 1 Processor -- start 2024-04-15T15:20:06.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.487+0000 7fa07be00640 1 -- start start 2024-04-15T15:20:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.488+0000 7fa07be00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa07c162020 0x7fa07c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.489+0000 7fa07be00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0cf110 0x7fa07c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.489+0000 7fa07be00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa07c0d0090 con 0x7fa07c0d1ac0 2024-04-15T15:20:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.489+0000 7fa07be00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fa07c0d1f50 con 0x7fa07c162020 2024-04-15T15:20:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.489+0000 7fa07be00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa07c163440 con 0x7fa07c0cf110 2024-04-15T15:20:06.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.489+0000 7fa07aa00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa07c162020 0x7fa07c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.489+0000 7fa07a000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0cf110 0x7fa07c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.490+0000 7fa07aa00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa07c162020 0x7fa07c162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:44856/0 (socket says 172.21.15.49:44856) 2024-04-15T15:20:06.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.490+0000 7fa07aa00640 1 -- 172.21.15.49:0/699377894 learned_addr learned my addr 172.21.15.49:0/699377894 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:06.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.490+0000 7fa07a000640 1 -- 172.21.15.49:0/699377894 >> v2:172.21.15.107:3300/0 conn(0x7fa07c162020 msgr2=0x7fa07c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:06.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.490+0000 7fa07a000640 1 --2- 172.21.15.49:0/699377894 >> v2:172.21.15.107:3300/0 conn(0x7fa07c162020 0x7fa07c162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.490+0000 7fa07a000640 1 -- 172.21.15.49:0/699377894 >> v1:172.21.15.107:6789/0 conn(0x7fa07c0d1ac0 legacy=0x7fa07c0b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.490+0000 7fa07a000640 1 -- 172.21.15.49:0/699377894 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07c0cfad0 con 0x7fa07c0cf110 2024-04-15T15:20:06.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.490+0000 7fa07aa00640 1 --2- 172.21.15.49:0/699377894 >> v2:172.21.15.107:3300/0 conn(0x7fa07c162020 0x7fa07c162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:06.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.492+0000 7fa07a000640 1 --2- 172.21.15.49:0/699377894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0cf110 0x7fa07c0d1580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa0680097a0 tx=0x7fa06802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd630348a950c1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.492+0000 7fa079600640 1 -- 172.21.15.49:0/699377894 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068035020 con 0x7fa07c0cf110 2024-04-15T15:20:06.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.492+0000 7fa079600640 1 -- 172.21.15.49:0/699377894 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa068031830 con 0x7fa07c0cf110 2024-04-15T15:20:06.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.492+0000 7fa079600640 1 -- 172.21.15.49:0/699377894 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068031c00 con 0x7fa07c0cf110 2024-04-15T15:20:06.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.493+0000 7fa07be00640 1 -- 172.21.15.49:0/699377894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0cf110 msgr2=0x7fa07c0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.493+0000 7fa07be00640 1 --2- 172.21.15.49:0/699377894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0cf110 0x7fa07c0d1580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa0680097a0 tx=0x7fa06802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.493+0000 7fa07be00640 1 -- 172.21.15.49:0/699377894 shutdown_connections 2024-04-15T15:20:06.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.493+0000 7fa07be00640 1 --2- 172.21.15.49:0/699377894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0cf110 0x7fa07c0d1580 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.493+0000 7fa07be00640 1 --2- 172.21.15.49:0/699377894 >> v2:172.21.15.107:3300/0 conn(0x7fa07c162020 0x7fa07c162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.493+0000 7fa07be00640 1 -- 172.21.15.49:0/699377894 >> 172.21.15.49:0/699377894 conn(0x7fa07c06cf00 msgr2=0x7fa07c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:06.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.493+0000 7fa07be00640 1 -- 172.21.15.49:0/699377894 shutdown_connections 2024-04-15T15:20:06.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.493+0000 7fa07be00640 1 -- 172.21.15.49:0/699377894 wait complete. 2024-04-15T15:20:06.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.495+0000 7fa07be00640 1 Processor -- start 2024-04-15T15:20:06.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.495+0000 7fa07be00640 1 -- start start 2024-04-15T15:20:06.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.496+0000 7fa07be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa07c0cf110 0x7fa07c154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.496+0000 7fa07be00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0d1ac0 0x7fa07c15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.496+0000 7fa07be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa07c162020 0x7fa07c15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.496+0000 7fa07be00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c0cf880 con 0x7fa07c0cf110 2024-04-15T15:20:06.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.496+0000 7fa07be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c0b7b30 con 0x7fa07c162020 2024-04-15T15:20:06.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.496+0000 7fa07be00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa07c0d2b90 con 0x7fa07c0d1ac0 2024-04-15T15:20:06.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.496+0000 7fa07aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa07c0cf110 0x7fa07c154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.497+0000 7fa07aa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa07c0cf110 0x7fa07c154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42870/0 (socket says 172.21.15.49:42870) 2024-04-15T15:20:06.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.497+0000 7fa07aa00640 1 -- 172.21.15.49:0/2478039769 learned_addr learned my addr 172.21.15.49:0/2478039769 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:06.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.497+0000 7fa07a000640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0d1ac0 0x7fa07c15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.497+0000 7fa07aa00640 1 -- 172.21.15.49:0/2478039769 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0d1ac0 msgr2=0x7fa07c15e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.497+0000 7fa07aa00640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0d1ac0 0x7fa07c15e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.497+0000 7fa07aa00640 1 -- 172.21.15.49:0/2478039769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa07c162020 msgr2=0x7fa07c15ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.497+0000 7fa07aa00640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa07c162020 0x7fa07c15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.497+0000 7fa07aa00640 1 -- 172.21.15.49:0/2478039769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07c15f180 con 0x7fa07c0cf110 2024-04-15T15:20:06.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.497+0000 7fa07aa00640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa07c0cf110 0x7fa07c154f70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00e9d0 tx=0x7fa06c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.498+0000 7fa072a00640 1 -- 172.21.15.49:0/2478039769 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c017070 con 0x7fa07c0cf110 2024-04-15T15:20:06.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.498+0000 7fa072a00640 1 -- 172.21.15.49:0/2478039769 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa06c00ca60 con 0x7fa07c0cf110 2024-04-15T15:20:06.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.498+0000 7fa072a00640 1 -- 172.21.15.49:0/2478039769 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00ce10 con 0x7fa07c0cf110 2024-04-15T15:20:06.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.498+0000 7fa07be00640 1 -- 172.21.15.49:0/2478039769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07c1e8470 con 0x7fa07c0cf110 2024-04-15T15:20:06.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.498+0000 7fa07be00640 1 -- 172.21.15.49:0/2478039769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07c1e8a30 con 0x7fa07c0cf110 2024-04-15T15:20:06.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.499+0000 7fa07be00640 1 -- 172.21.15.49:0/2478039769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07c0755c0 con 0x7fa07c0cf110 2024-04-15T15:20:06.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.500+0000 7fa072a00640 1 -- 172.21.15.49:0/2478039769 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa06c01a070 con 0x7fa07c0cf110 2024-04-15T15:20:06.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.501+0000 7fa072a00640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa05c06c8f0 0x7fa05c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:06.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.501+0000 7fa072a00640 1 -- 172.21.15.49:0/2478039769 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa06c08bf30 con 0x7fa07c0cf110 2024-04-15T15:20:06.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.501+0000 7fa07a000640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa05c06c8f0 0x7fa05c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:06.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.502+0000 7fa07a000640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa05c06c8f0 0x7fa05c06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa068036000 tx=0x7fa068008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:06.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.506+0000 7fa072a00640 1 -- 172.21.15.49:0/2478039769 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa06c05a2c0 con 0x7fa07c0cf110 2024-04-15T15:20:06.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.751+0000 7fa07be00640 1 -- 172.21.15.49:0/2478039769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa07c0b7650 con 0x7fa07c0cf110 2024-04-15T15:20:06.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.752+0000 7fa072a00640 1 -- 172.21.15.49:0/2478039769 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7fa06c059bc0 con 0x7fa07c0cf110 2024-04-15T15:20:06.754 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:06.754 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:00.037472+0000","modified":"2024-04-15T15:20:03.151587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5067":{"gid":5067,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/1023024717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1023024717},{"type":"v1","addr":"172.21.15.49:6837","nonce":1023024717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5067,"qdb_cluster":[5067]},"id":2}]} 2024-04-15T15:20:06.755 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-04-15T15:20:06.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 -- 172.21.15.49:0/2478039769 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa05c06c8f0 msgr2=0x7fa05c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa05c06c8f0 0x7fa05c06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa068036000 tx=0x7fa068008030 comp rx=0 tx=0).stop 2024-04-15T15:20:06.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 -- 172.21.15.49:0/2478039769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa07c0cf110 msgr2=0x7fa07c154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:06.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa07c0cf110 0x7fa07c154f70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00e9d0 tx=0x7fa06c00ee90 comp rx=0 tx=0).stop 2024-04-15T15:20:06.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 -- 172.21.15.49:0/2478039769 shutdown_connections 2024-04-15T15:20:06.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa05c06c8f0 0x7fa05c06edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07c0d1ac0 0x7fa07c15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa07c162020 0x7fa07c15ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 --2- 172.21.15.49:0/2478039769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa07c0cf110 0x7fa07c154f70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:06.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 -- 172.21.15.49:0/2478039769 >> 172.21.15.49:0/2478039769 conn(0x7fa07c06cf00 msgr2=0x7fa07c0d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:06.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.755+0000 7fa07be00640 1 -- 172.21.15.49:0/2478039769 shutdown_connections 2024-04-15T15:20:06.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:06.756+0000 7fa07be00640 1 -- 172.21.15.49:0/2478039769 wait complete. 2024-04-15T15:20:07.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:20:00.037472+0000', 'modified': '2024-04-15T15:20:03.151587+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5067}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5067': {'gid': 5067, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6837/1023024717', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 1023024717}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 1023024717}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5067, 'qdb_cluster': [5067]} 2024-04-15T15:20:07.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-15T15:20:07.180 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-15T15:20:07.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.425+0000 7fb7c7200640 1 Processor -- start 2024-04-15T15:20:07.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.425+0000 7fb7c7200640 1 -- start start 2024-04-15T15:20:07.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.427+0000 7fb7c7200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb7c815eab0 0x7fb7c8162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.427+0000 7fb7c7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7c8162b70 0x7fb7c80d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.427+0000 7fb7c7200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7c80d1a90 0x7fb7c80b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.427+0000 7fb7c7200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb7c80d2e50 con 0x7fb7c80d1a90 2024-04-15T15:20:07.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.427+0000 7fb7c7200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fb7c80b78f0 con 0x7fb7c815eab0 2024-04-15T15:20:07.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.427+0000 7fb7c7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c8163010 con 0x7fb7c8162b70 2024-04-15T15:20:07.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.427+0000 7fb7c6800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7c80d1a90 0x7fb7c80b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.427+0000 7fb7c6800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7c80d1a90 0x7fb7c80b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42876/0 (socket says 172.21.15.49:42876) 2024-04-15T15:20:07.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.427+0000 7fb7c6800640 1 -- 172.21.15.49:0/2826052814 learned_addr learned my addr 172.21.15.49:0/2826052814 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:07.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.428+0000 7fb7c6800640 1 -- 172.21.15.49:0/2826052814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7c8162b70 msgr2=0x7fb7c80d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.428+0000 7fb7c6800640 1 --2- 172.21.15.49:0/2826052814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7c8162b70 0x7fb7c80d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.428+0000 7fb7c6800640 1 -- 172.21.15.49:0/2826052814 >> v2:172.21.15.107:3300/0 conn(0x7fb7c815eab0 msgr2=0x7fb7c8162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.428+0000 7fb7c6800640 1 --2- 172.21.15.49:0/2826052814 >> v2:172.21.15.107:3300/0 conn(0x7fb7c815eab0 0x7fb7c8162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.428+0000 7fb7c6800640 1 -- 172.21.15.49:0/2826052814 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c81632f0 con 0x7fb7c80d1a90 2024-04-15T15:20:07.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.429+0000 7fb7c6800640 1 --2- 172.21.15.49:0/2826052814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c80d1a90 0x7fb7c80b6240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb7c808d2c0 tx=0x7fb7c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72958c19a627c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.429+0000 7fb7c5400640 1 -- 172.21.15.49:0/2826052814 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c002ca70 con 0x7fb7c80d1a90 2024-04-15T15:20:07.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.429+0000 7fb7c5400640 1 -- 172.21.15.49:0/2826052814 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c002cc10 con 0x7fb7c80d1a90 2024-04-15T15:20:07.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.429+0000 7fb7c5400640 1 -- 172.21.15.49:0/2826052814 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00343f0 con 0x7fb7c80d1a90 2024-04-15T15:20:07.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.430+0000 7fb7c7200640 1 -- 172.21.15.49:0/2826052814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c80d1a90 msgr2=0x7fb7c80b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.430+0000 7fb7c7200640 1 --2- 172.21.15.49:0/2826052814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c80d1a90 0x7fb7c80b6240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb7c808d2c0 tx=0x7fb7c002bed0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.430+0000 7fb7c7200640 1 -- 172.21.15.49:0/2826052814 shutdown_connections 2024-04-15T15:20:07.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.430+0000 7fb7c7200640 1 --2- 172.21.15.49:0/2826052814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c80d1a90 0x7fb7c80b6240 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.430+0000 7fb7c7200640 1 --2- 172.21.15.49:0/2826052814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7c8162b70 0x7fb7c80d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.430+0000 7fb7c7200640 1 --2- 172.21.15.49:0/2826052814 >> v2:172.21.15.107:3300/0 conn(0x7fb7c815eab0 0x7fb7c8162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.430+0000 7fb7c7200640 1 -- 172.21.15.49:0/2826052814 >> 172.21.15.49:0/2826052814 conn(0x7fb7c8073240 msgr2=0x7fb7c8073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.430+0000 7fb7c7200640 1 -- 172.21.15.49:0/2826052814 shutdown_connections 2024-04-15T15:20:07.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.430+0000 7fb7c7200640 1 -- 172.21.15.49:0/2826052814 wait complete. 2024-04-15T15:20:07.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.432+0000 7fb7c7200640 1 Processor -- start 2024-04-15T15:20:07.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.432+0000 7fb7c7200640 1 -- start start 2024-04-15T15:20:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.433+0000 7fb7c7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7c80d1a90 0x7fb7c81527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.433+0000 7fb7c7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c815eab0 0x7fb7c81f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.433+0000 7fb7c7200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c8162b70 0x7fb7c815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.433+0000 7fb7c7200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c80ba590 con 0x7fb7c8162b70 2024-04-15T15:20:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.433+0000 7fb7c7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c815ee90 con 0x7fb7c815eab0 2024-04-15T15:20:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.433+0000 7fb7c7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c80ba710 con 0x7fb7c80d1a90 2024-04-15T15:20:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.433+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c8162b70 0x7fb7c815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.433+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c8162b70 0x7fb7c815f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42886/0 (socket says 172.21.15.49:42886) 2024-04-15T15:20:07.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.433+0000 7fb7c6800640 1 -- 172.21.15.49:0/1136058320 learned_addr learned my addr 172.21.15.49:0/1136058320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7c6800640 1 -- 172.21.15.49:0/1136058320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7c80d1a90 msgr2=0x7fb7c81527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7bfe00640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c815eab0 0x7fb7c81f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7c6800640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7c80d1a90 0x7fb7c81527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7c6800640 1 -- 172.21.15.49:0/1136058320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c815eab0 msgr2=0x7fb7c81f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7c6800640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c815eab0 0x7fb7c81f5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7c6800640 1 -- 172.21.15.49:0/1136058320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c815f550 con 0x7fb7c8162b70 2024-04-15T15:20:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7c6800640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c8162b70 0x7fb7c815f010 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0036040 tx=0x7fb7c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7be000640 1 -- 172.21.15.49:0/1136058320 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0043020 con 0x7fb7c8162b70 2024-04-15T15:20:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7be000640 1 -- 172.21.15.49:0/1136058320 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c003e450 con 0x7fb7c8162b70 2024-04-15T15:20:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7be000640 1 -- 172.21.15.49:0/1136058320 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c003e820 con 0x7fb7c8162b70 2024-04-15T15:20:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.434+0000 7fb7c7200640 1 -- 172.21.15.49:0/1136058320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c815f7d0 con 0x7fb7c8162b70 2024-04-15T15:20:07.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.435+0000 7fb7c7200640 1 -- 172.21.15.49:0/1136058320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c815fd90 con 0x7fb7c8162b70 2024-04-15T15:20:07.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.438+0000 7fb7be000640 1 -- 172.21.15.49:0/1136058320 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7c0046020 con 0x7fb7c8162b70 2024-04-15T15:20:07.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.438+0000 7fb7be000640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb7a406c8f0 0x7fb7a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.439+0000 7fb7be000640 1 -- 172.21.15.49:0/1136058320 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb7c00ae530 con 0x7fb7c8162b70 2024-04-15T15:20:07.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.439+0000 7fb7c5e00640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb7a406c8f0 0x7fb7a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.439+0000 7fb7cf9d4740 1 Processor -- start 2024-04-15T15:20:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.439+0000 7fb7cf9d4740 1 -- start start 2024-04-15T15:20:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.440+0000 7fb7c5e00640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb7a406c8f0 0x7fb7a406edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009710 tx=0x7fb7b8006c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.440+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef37e0 0x559142f12920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.440+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142ef3bc0 0x559142ecc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.440+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ecca00 0x559142eca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.440+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559142e319c0 con 0x559142ef3bc0 2024-04-15T15:20:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.440+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559142dd8560 con 0x559142ecca00 2024-04-15T15:20:07.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.440+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x559142e2d980 con 0x559142ef37e0 2024-04-15T15:20:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.440+0000 7fb7bfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142ef3bc0 0x559142ecc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.441+0000 7fb7bfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142ef3bc0 0x559142ecc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42894/0 (socket says 172.21.15.49:42894) 2024-04-15T15:20:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.441+0000 7fb7bfe00640 1 -- 172.21.15.49:0/1333373909 learned_addr learned my addr 172.21.15.49:0/1333373909 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.441+0000 7fb7c6800640 1 --2- 172.21.15.49:0/1333373909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ecca00 0x559142eca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.441+0000 7fb7bfe00640 1 -- 172.21.15.49:0/1333373909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef37e0 msgr2=0x559142f12920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.441+0000 7fb7bfe00640 1 --2- 172.21.15.49:0/1333373909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef37e0 0x559142f12920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.441+0000 7fb7bfe00640 1 -- 172.21.15.49:0/1333373909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ecca00 msgr2=0x559142eca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.441+0000 7fb7bfe00640 1 --2- 172.21.15.49:0/1333373909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ecca00 0x559142eca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.441+0000 7fb7bfe00640 1 -- 172.21.15.49:0/1333373909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559142b442d0 con 0x559142ef3bc0 2024-04-15T15:20:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.441+0000 7fb7bfe00640 1 --2- 172.21.15.49:0/1333373909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142ef3bc0 0x559142ecc4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000a7b0 tx=0x7fb7b000ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2785c3b0dff8de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7b6600640 1 -- 172.21.15.49:0/1333373909 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0015070 con 0x559142ef3bc0 2024-04-15T15:20:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7b6600640 1 -- 172.21.15.49:0/1333373909 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b000cae0 con 0x559142ef3bc0 2024-04-15T15:20:07.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7b6600640 1 -- 172.21.15.49:0/1333373909 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000ae60 con 0x559142ef3bc0 2024-04-15T15:20:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/1333373909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142ef3bc0 msgr2=0x559142ecc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/1333373909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142ef3bc0 0x559142ecc4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000a7b0 tx=0x7fb7b000ac70 comp rx=0 tx=0).stop 2024-04-15T15:20:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/1333373909 shutdown_connections 2024-04-15T15:20:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/1333373909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ecca00 0x559142eca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/1333373909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142ef3bc0 0x559142ecc4c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/1333373909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef37e0 0x559142f12920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/1333373909 >> 172.21.15.49:0/1333373909 conn(0x559142ec8aa0 msgr2=0x559142f11620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/1333373909 shutdown_connections 2024-04-15T15:20:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.442+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/1333373909 wait complete. 2024-04-15T15:20:07.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.443+0000 7fb7cf9d4740 1 -- start start 2024-04-15T15:20:07.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.443+0000 7fb7cf9d4740 1 Processor -- start 2024-04-15T15:20:07.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.444+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef37e0 0x559142ef5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.444+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef3bc0 0x559142f0cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.444+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142eef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.444+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559142ec68d0 con 0x559142f0d4f0 2024-04-15T15:20:07.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.444+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559142deb920 con 0x559142ef37e0 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.444+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x559142ecce80 con 0x559142ef3bc0 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.444+0000 7fb7cf9d4740 10 client.0 _ll_register_callbacks cb 0x559142deb310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.445+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142eef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.445+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142eef830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42904/0 (socket says 172.21.15.49:42904) 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.445+0000 7fb7c6800640 1 -- 172.21.15.49:0/2874126195 learned_addr learned my addr 172.21.15.49:0/2874126195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.445+0000 7fb7c6800640 1 -- 172.21.15.49:0/2874126195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef3bc0 msgr2=0x559142f0cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.445+0000 7fb7c6800640 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef3bc0 0x559142f0cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.445+0000 7fb7c6800640 1 -- 172.21.15.49:0/2874126195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef37e0 msgr2=0x559142ef5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.445+0000 7fb7c6800640 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef37e0 0x559142ef5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.445+0000 7fb7c6800640 1 -- 172.21.15.49:0/2874126195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559142c0e6b0 con 0x559142f0d4f0 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.445+0000 7fb7c6800640 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142eef830 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7c007c150 tx=0x7fb7c007c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de9207851036971 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.445+0000 7fb7aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-15T15:20:07.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2874126195 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0084da0 con 0x559142f0d4f0 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2874126195 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb7c0080170 con 0x559142f0d4f0 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2874126195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b000cae0 con 0x559142f0d4f0 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2874126195 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb7c0080370 con 0x559142f0d4f0 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7cf9d4740 10 client.5232 fetch_fsmap learned FSMap version 17 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2874126195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559142f0f6d0 con 0x559142f0d4f0 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2874126195 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b000cae0 con 0x559142f0d4f0 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2874126195 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fb7c007c420 con 0x559142f0d4f0 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7cf9d4740 10 client.5232 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7cf9d4740 10 client.5232 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7cf9d4740 20 client.5232 populate_metadata read hostname 'smithi049' 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x559142f25a80 0x559142f25e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7cf9d4740 4 client.5232 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.446+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2874126195 --> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559142dd7b70 con 0x559142f25a80 2024-04-15T15:20:07.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.447+0000 7fb7c5e00640 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x559142f25a80 0x559142f25e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.447+0000 7fb7c5e00640 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x559142f25a80 0x559142f25e60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59d09c09b65bebf server_cookie=269eaa34daf6bb96 in_seq=0 out_seq=0 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.447+0000 7fb7aaa00640 10 client.5232 ms_handle_connect on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.448+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2874126195 <== mds.0 v2:172.21.15.49:6836/1023024717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559142dd7b70 con 0x559142f25a80 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.448+0000 7fb7aaa00640 10 client.5232 handle_command_reply: tid=0 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.448+0000 7fb7cf9d4740 1 client.5232 shutdown 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.448+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2874126195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 msgr2=0x559142eef830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.448+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142eef830 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7c007c150 tx=0x7fb7c007c980 comp rx=0 tx=0).stop 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.449+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2874126195 shutdown_connections 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.449+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x559142f25a80 0x559142f25e60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.449+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef3bc0 0x559142f0cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.449+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142eef830 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.449+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2874126195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef37e0 0x559142ef5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.449+0000 7fb7aaa00640 0 client.5232 ms_handle_reset on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:07.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.449+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2874126195 >> 172.21.15.49:0/2874126195 conn(0x559142ecca00 msgr2=0x559142f115b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.449+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2874126195 shutdown_connections 2024-04-15T15:20:07.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.449+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2874126195 wait complete. 2024-04-15T15:20:07.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.449+0000 7fb7cf9d4740 20 client.5232 trim_cache size 0 max 16384 2024-04-15T15:20:07.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.469+0000 7fb7cf9d4740 1 Processor -- start 2024-04-15T15:20:07.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.469+0000 7fb7cf9d4740 1 -- start start 2024-04-15T15:20:07.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.470+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef37e0 0x559142f0ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.470+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef3bc0 0x559142f115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.470+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.470+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81ffe90 con 0x559142f0d4f0 2024-04-15T15:20:07.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.470+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81ff1f0 con 0x559142ef37e0 2024-04-15T15:20:07.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.470+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c81ff840 con 0x559142ef3bc0 2024-04-15T15:20:07.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.470+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.471+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42908/0 (socket says 172.21.15.49:42908) 2024-04-15T15:20:07.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.471+0000 7fb7c6800640 1 -- 172.21.15.49:0/1757855156 learned_addr learned my addr 172.21.15.49:0/1757855156 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:07.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.471+0000 7fb7c5e00640 1 --2- 172.21.15.49:0/1757855156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef37e0 0x559142f0ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.471+0000 7fb7bfe00640 1 --2- 172.21.15.49:0/1757855156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef3bc0 0x559142f115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.471+0000 7fb7c6800640 1 -- 172.21.15.49:0/1757855156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef3bc0 msgr2=0x559142f115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.471+0000 7fb7c6800640 1 --2- 172.21.15.49:0/1757855156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef3bc0 0x559142f115b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.471+0000 7fb7c6800640 1 -- 172.21.15.49:0/1757855156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef37e0 msgr2=0x559142f0ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.471+0000 7fb7c6800640 1 --2- 172.21.15.49:0/1757855156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef37e0 0x559142f0ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.471+0000 7fb7c6800640 1 -- 172.21.15.49:0/1757855156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559142ef5550 con 0x559142f0d4f0 2024-04-15T15:20:07.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.471+0000 7fb7c6800640 1 --2- 172.21.15.49:0/1757855156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11af0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00093e0 tx=0x7fb7c0049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c59170e48c496e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7b5c00640 1 -- 172.21.15.49:0/1757855156 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c002ca80 con 0x559142f0d4f0 2024-04-15T15:20:07.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7b5c00640 1 -- 172.21.15.49:0/1757855156 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c007c420 con 0x559142f0d4f0 2024-04-15T15:20:07.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7b5c00640 1 -- 172.21.15.49:0/1757855156 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0090df0 con 0x559142f0d4f0 2024-04-15T15:20:07.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/1757855156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 msgr2=0x559142f11af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/1757855156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11af0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00093e0 tx=0x7fb7c0049000 comp rx=0 tx=0).stop 2024-04-15T15:20:07.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/1757855156 shutdown_connections 2024-04-15T15:20:07.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/1757855156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef3bc0 0x559142f115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/1757855156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11af0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/1757855156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef37e0 0x559142f0ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/1757855156 >> 172.21.15.49:0/1757855156 conn(0x559142ecca00 msgr2=0x559142f0ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.472+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/1757855156 shutdown_connections 2024-04-15T15:20:07.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.473+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/1757855156 wait complete. 2024-04-15T15:20:07.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.473+0000 7fb7cf9d4740 1 -- start start 2024-04-15T15:20:07.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.473+0000 7fb7cf9d4740 1 Processor -- start 2024-04-15T15:20:07.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.474+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef37e0 0x559142ef1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.474+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef3bc0 0x559142f24540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.474+0000 7fb7cf9d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.474+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559142ec68d0 con 0x559142f0d4f0 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.474+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559142deb920 con 0x559142ef3bc0 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.474+0000 7fb7cf9d4740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x559142f10720 con 0x559142ef37e0 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.474+0000 7fb7cf9d4740 10 client.0 _ll_register_callbacks cb 0x7fb7c0009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7c6800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42918/0 (socket says 172.21.15.49:42918) 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7c6800640 1 -- 172.21.15.49:0/2694586096 learned_addr learned my addr 172.21.15.49:0/2694586096 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7c5e00640 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef37e0 0x559142ef1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7bfe00640 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef3bc0 0x559142f24540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7c6800640 1 -- 172.21.15.49:0/2694586096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef37e0 msgr2=0x559142ef1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7c6800640 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef37e0 0x559142ef1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7c6800640 1 -- 172.21.15.49:0/2694586096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef3bc0 msgr2=0x559142f24540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7c6800640 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef3bc0 0x559142f24540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7c6800640 1 -- 172.21.15.49:0/2694586096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559142f23db0 con 0x559142f0d4f0 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7bfe00640 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef3bc0 0x559142f24540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7c6800640 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11380 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb7c007c150 tx=0x7fb7c00ac890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcaca9fb6524ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.475+0000 7fb7aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2694586096 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00a1060 con 0x559142f0d4f0 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2694586096 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb7c0084b20 con 0x559142f0d4f0 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2694586096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c007c420 con 0x559142f0d4f0 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2694586096 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb7c00ae070 con 0x559142f0d4f0 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7cf9d4740 10 client.5238 fetch_fsmap learned FSMap version 17 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2694586096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559142ef2690 con 0x559142f0d4f0 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2694586096 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c007c420 con 0x559142f0d4f0 2024-04-15T15:20:07.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2694586096 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fb7c007b610 con 0x559142f0d4f0 2024-04-15T15:20:07.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7cf9d4740 10 client.5238 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:20:07.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7cf9d4740 10 client.5238 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-15T15:20:07.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7cf9d4740 20 client.5238 populate_metadata read hostname 'smithi049' 2024-04-15T15:20:07.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.476+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x559142f0e370 0x559142f118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.477+0000 7fb7cf9d4740 4 client.5238 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-15T15:20:07.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.477+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2694586096 --> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] -- command(tid 0: {"prefix": "status"}) -- 0x559142e3f3e0 con 0x559142f0e370 2024-04-15T15:20:07.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.477+0000 7fb7c5e00640 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x559142f0e370 0x559142f118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.477+0000 7fb7c5e00640 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x559142f0e370 0x559142f118c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20689be866056d9a server_cookie=670f782fb4fabe30 in_seq=0 out_seq=0 2024-04-15T15:20:07.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.477+0000 7fb7aaa00640 10 client.5238 ms_handle_connect on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:07.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7aaa00640 1 -- 172.21.15.49:0/2694586096 <== mds.0 v2:172.21.15.49:6836/1023024717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x559142e3f3e0 con 0x559142f0e370 2024-04-15T15:20:07.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7aaa00640 10 client.5238 handle_command_reply: tid=0 2024-04-15T15:20:07.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 client.5238 shutdown 2024-04-15T15:20:07.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2694586096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 msgr2=0x559142f11380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11380 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb7c007c150 tx=0x7fb7c00ac890 comp rx=0 tx=0).stop 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2694586096 shutdown_connections 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x559142f0e370 0x559142f118c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559142f0d4f0 0x559142f11380 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559142ef3bc0 0x559142f24540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 --2- 172.21.15.49:0/2694586096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559142ef37e0 0x559142ef1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7aaa00640 0 client.5238 ms_handle_reset on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2694586096 >> 172.21.15.49:0/2694586096 conn(0x559142ecca00 msgr2=0x559142f35ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2694586096 shutdown_connections 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 1 -- 172.21.15.49:0/2694586096 wait complete. 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.478+0000 7fb7cf9d4740 20 client.5238 trim_cache size 0 max 16384 2024-04-15T15:20:07.483 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c", 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 5067, 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 6.8816202439999996, 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 17, 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 35, 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 35, 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 15.916466228999999 2024-04-15T15:20:07.484 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:20:07.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 -- 172.21.15.49:0/1136058320 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb7a406c8f0 msgr2=0x7fb7a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb7a406c8f0 0x7fb7a406edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009710 tx=0x7fb7b8006c90 comp rx=0 tx=0).stop 2024-04-15T15:20:07.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 -- 172.21.15.49:0/1136058320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c8162b70 msgr2=0x7fb7c815f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c8162b70 0x7fb7c815f010 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0036040 tx=0x7fb7c0004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:07.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 -- 172.21.15.49:0/1136058320 shutdown_connections 2024-04-15T15:20:07.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb7a406c8f0 0x7fb7a406edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c8162b70 0x7fb7c815f010 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c815eab0 0x7fb7c81f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 --2- 172.21.15.49:0/1136058320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7c80d1a90 0x7fb7c81527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 -- 172.21.15.49:0/1136058320 >> 172.21.15.49:0/1136058320 conn(0x7fb7c8073240 msgr2=0x7fb7c80c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 -- 172.21.15.49:0/1136058320 shutdown_connections 2024-04-15T15:20:07.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.479+0000 7fb7c7200640 1 -- 172.21.15.49:0/1136058320 wait complete. 2024-04-15T15:20:07.501 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:07.632 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:20:07.640 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:20:07.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.716+0000 7f562d800640 1 Processor -- start 2024-04-15T15:20:07.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.716+0000 7f562d800640 1 -- start start 2024-04-15T15:20:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.718+0000 7f562d800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f56280b6af0 0x7f56280d4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.718+0000 7f562d800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56280b7a90 0x7f56280b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.718+0000 7f562d800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56280b41d0 con 0x7f56280b3df0 2024-04-15T15:20:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.718+0000 7f562d800640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5628162780 con 0x7f56280b7a90 2024-04-15T15:20:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.718+0000 7f562d800640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f56280cfa40 con 0x7f56280b6af0 2024-04-15T15:20:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.718+0000 7f5627400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56280b7a90 0x7f56280b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.718+0000 7f5627400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56280b7a90 0x7f56280b7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42922/0 (socket says 172.21.15.49:42922) 2024-04-15T15:20:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.718+0000 7f5627400640 1 -- 172.21.15.49:0/1977697328 learned_addr learned my addr 172.21.15.49:0/1977697328 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.718+0000 7f5626000640 1 --2- 172.21.15.49:0/1977697328 >> v2:172.21.15.107:3300/0 conn(0x7f56280b6af0 0x7f56280d4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.719+0000 7f5627400640 1 -- 172.21.15.49:0/1977697328 >> v2:172.21.15.107:3300/0 conn(0x7f56280b6af0 msgr2=0x7f56280d4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:07.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.719+0000 7f5627400640 1 --2- 172.21.15.49:0/1977697328 >> v2:172.21.15.107:3300/0 conn(0x7f56280b6af0 0x7f56280d4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.719+0000 7f5627400640 1 -- 172.21.15.49:0/1977697328 >> v1:172.21.15.107:6789/0 conn(0x7f56280b3df0 legacy=0x7f56280b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.719+0000 7f5627400640 1 -- 172.21.15.49:0/1977697328 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56280cfce0 con 0x7f56280b7a90 2024-04-15T15:20:07.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.720+0000 7f5627400640 1 --2- 172.21.15.49:0/1977697328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b7a90 0x7f56280b7ea0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f562806e020 tx=0x7f562002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a73231bd07820c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.720+0000 7f5625600640 1 -- 172.21.15.49:0/1977697328 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562002cb20 con 0x7f56280b7a90 2024-04-15T15:20:07.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.720+0000 7f5625600640 1 -- 172.21.15.49:0/1977697328 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562002ccc0 con 0x7f56280b7a90 2024-04-15T15:20:07.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.720+0000 7f5625600640 1 -- 172.21.15.49:0/1977697328 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620034610 con 0x7f56280b7a90 2024-04-15T15:20:07.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.721+0000 7f562d800640 1 -- 172.21.15.49:0/1977697328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b7a90 msgr2=0x7f56280b7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.721+0000 7f562d800640 1 --2- 172.21.15.49:0/1977697328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b7a90 0x7f56280b7ea0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f562806e020 tx=0x7f562002bf80 comp rx=0 tx=0).stop 2024-04-15T15:20:07.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.721+0000 7f562d800640 1 -- 172.21.15.49:0/1977697328 shutdown_connections 2024-04-15T15:20:07.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.721+0000 7f562d800640 1 --2- 172.21.15.49:0/1977697328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b7a90 0x7f56280b7ea0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.721+0000 7f562d800640 1 --2- 172.21.15.49:0/1977697328 >> v2:172.21.15.107:3300/0 conn(0x7f56280b6af0 0x7f56280d4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.721+0000 7f562d800640 1 -- 172.21.15.49:0/1977697328 >> 172.21.15.49:0/1977697328 conn(0x7f562806d5f0 msgr2=0x7f56280d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.721+0000 7f562d800640 1 -- 172.21.15.49:0/1977697328 shutdown_connections 2024-04-15T15:20:07.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.721+0000 7f562d800640 1 -- 172.21.15.49:0/1977697328 wait complete. 2024-04-15T15:20:07.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.723+0000 7f562d800640 1 Processor -- start 2024-04-15T15:20:07.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.723+0000 7f562d800640 1 -- start start 2024-04-15T15:20:07.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.724+0000 7f562d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56280b3df0 0x7f56280b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.724+0000 7f562d800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b6af0 0x7f562814eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.724+0000 7f562d800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56280b7a90 0x7f562814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.724+0000 7f562d800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56280cffc0 con 0x7f56280b6af0 2024-04-15T15:20:07.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.724+0000 7f562d800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f56280d22b0 con 0x7f56280b3df0 2024-04-15T15:20:07.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.724+0000 7f562d800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f56280d3500 con 0x7f56280b7a90 2024-04-15T15:20:07.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.724+0000 7f5626000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b6af0 0x7f562814eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.724+0000 7f5626000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b6af0 0x7f562814eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42938/0 (socket says 172.21.15.49:42938) 2024-04-15T15:20:07.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.724+0000 7f5626000640 1 -- 172.21.15.49:0/941583655 learned_addr learned my addr 172.21.15.49:0/941583655 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.724+0000 7f5626a00640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56280b3df0 0x7f56280b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.725+0000 7f5626000640 1 -- 172.21.15.49:0/941583655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56280b7a90 msgr2=0x7f562814f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.725+0000 7f5626000640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56280b7a90 0x7f562814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.725+0000 7f5626000640 1 -- 172.21.15.49:0/941583655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56280b3df0 msgr2=0x7f56280b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.725+0000 7f5626000640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56280b3df0 0x7f56280b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.725+0000 7f5626000640 1 -- 172.21.15.49:0/941583655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56280c9270 con 0x7f56280b6af0 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.725+0000 7f5626000640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b6af0 0x7f562814eba0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f561400e9b0 tx=0x7f561400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.725+0000 7f561ea00640 1 -- 172.21.15.49:0/941583655 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614017070 con 0x7f56280b6af0 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.725+0000 7f561ea00640 1 -- 172.21.15.49:0/941583655 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561400c9a0 con 0x7f56280b6af0 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.725+0000 7f561ea00640 1 -- 172.21.15.49:0/941583655 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561400cd90 con 0x7f56280b6af0 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.725+0000 7f562d800640 1 -- 172.21.15.49:0/941583655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56280c9550 con 0x7f56280b6af0 2024-04-15T15:20:07.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.726+0000 7f562d800640 1 -- 172.21.15.49:0/941583655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56280c9b10 con 0x7f56280b6af0 2024-04-15T15:20:07.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.727+0000 7f562d800640 1 -- 172.21.15.49:0/941583655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5628068c80 con 0x7f56280b6af0 2024-04-15T15:20:07.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.728+0000 7f561ea00640 1 -- 172.21.15.49:0/941583655 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f561401a070 con 0x7f56280b6af0 2024-04-15T15:20:07.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.729+0000 7f561ea00640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f560806c8f0 0x7f560806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:07.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.729+0000 7f561ea00640 1 -- 172.21.15.49:0/941583655 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f561408bf80 con 0x7f56280b6af0 2024-04-15T15:20:07.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.729+0000 7f5626a00640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f560806c8f0 0x7f560806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:07.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.730+0000 7f5626a00640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f560806c8f0 0x7f560806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5618007fd0 tx=0x7f5618009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:07.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.734+0000 7f561ea00640 1 -- 172.21.15.49:0/941583655 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f561405a310 con 0x7f56280b6af0 2024-04-15T15:20:07.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.985+0000 7f562d800640 1 -- 172.21.15.49:0/941583655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f56280b7070 con 0x7f56280b6af0 2024-04-15T15:20:07.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.986+0000 7f561ea00640 1 -- 172.21.15.49:0/941583655 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f5614059c10 con 0x7f56280b6af0 2024-04-15T15:20:07.988 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:07.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 -- 172.21.15.49:0/941583655 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f560806c8f0 msgr2=0x7f560806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f560806c8f0 0x7f560806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5618007fd0 tx=0x7f5618009040 comp rx=0 tx=0).stop 2024-04-15T15:20:07.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 -- 172.21.15.49:0/941583655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b6af0 msgr2=0x7f562814eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:07.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b6af0 0x7f562814eba0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f561400e9b0 tx=0x7f561400ee70 comp rx=0 tx=0).stop 2024-04-15T15:20:07.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 -- 172.21.15.49:0/941583655 shutdown_connections 2024-04-15T15:20:07.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f560806c8f0 0x7f560806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56280b7a90 0x7f562814f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56280b6af0 0x7f562814eba0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 --2- 172.21.15.49:0/941583655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56280b3df0 0x7f56280b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:07.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 -- 172.21.15.49:0/941583655 >> 172.21.15.49:0/941583655 conn(0x7f562806d5f0 msgr2=0x7f5628161f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:07.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 -- 172.21.15.49:0/941583655 shutdown_connections 2024-04-15T15:20:07.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:07.989+0000 7f562d800640 1 -- 172.21.15.49:0/941583655 wait complete. 2024-04-15T15:20:08.009 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:20:08.009 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:08.009 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-04-15T15:20:08.009 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-15T15:20:08.010 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:08.010 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:20:08.010 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-15T15:20:08.010 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:08.010 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:08.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:08.010 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:08.066 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42495sec preferred_lft 42495sec 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 3a:d8:a3:bb:09:2d brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:08.067 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:08.068 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:08.068 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::6c47:44ff:feb2:eb9c/64 scope link 2024-04-15T15:20:08.068 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:08.068 INFO:teuthology.orchestra.run.smithi049.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:08.068 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 3a:d8:a3:bb:09:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:08.068 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::38d8:a3ff:febb:92d/64 scope link 2024-04-15T15:20:08.068 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:08.068 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:08.068 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 66:4c:0b:61:1e:4f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:08.068 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::644c:bff:fe61:1e4f/64 scope link 2024-04-15T15:20:08.069 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:08.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:08.070 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:08.169 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-04-15T15:20:08.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:08.173 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-04-15T15:20:08.230 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-04-15T15:20:08.231 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:08.231 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-15T15:20:08.231 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-15T15:20:08.231 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:08.232 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:08.232 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:20:08.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-15T15:20:08.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-15T15:20:08.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-15T15:20:08.232 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:20:08.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:08.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:08.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:08.516 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-04-15T15:20:08.573 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:20:08.573 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:20:08.574 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:08.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:08.574 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-04-15T15:20:08.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:08.678 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:08.733 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:08.733 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:08.733 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:08.788 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:20:08.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:08.788 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:20:08.938 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:20:08.938 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:20:08.938 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:20:08.961 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:20:08.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:20:08.961 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:20:08.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:08.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:08.988 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:20:08.988 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:20:08.988 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:20:08.988 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:08.989 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:08.989 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790" 2024-04-15T15:20:08.989 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790:/ 2024-04-15T15:20:08.989 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c" will pass to kernel 2024-04-15T15:20:08.990 INFO:teuthology.orchestra.run.smithi049.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-15T15:20:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:20:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:20:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:20:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:20:08.994 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:20:08.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:08.994 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:09.086 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:20:09.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:20:09.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:09.222 INFO:teuthology.orchestra.run.smithi049.stdout:client4730 2024-04-15T15:20:09.222 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:20:09.222 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:09.277 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.502+0000 7fd4b7a00640 1 Processor -- start 2024-04-15T15:20:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.502+0000 7fd4b7a00640 1 -- start start 2024-04-15T15:20:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.504+0000 7fd4b7a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4b01461f0 0x7fd4b00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.504+0000 7fd4b7a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4b00b4be0 con 0x7fd4b00b3fe0 2024-04-15T15:20:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.504+0000 7fd4b7a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4b00b4620 con 0x7fd4b00cedf0 2024-04-15T15:20:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.504+0000 7fd4b7a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd4b00a9640 con 0x7fd4b01461f0 2024-04-15T15:20:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.504+0000 7fd4b5000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4b01461f0 0x7fd4b00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.504+0000 7fd4b5a00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd4b00b3fe0 0x7fd4b015d7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:40404/0 (socket says 172.21.15.49:40404) 2024-04-15T15:20:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.504+0000 7fd4b5a00640 1 -- 172.21.15.49:0/276017842 learned_addr learned my addr 172.21.15.49:0/276017842 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.505+0000 7fd4b5000640 1 -- 172.21.15.49:0/276017842 >> v1:172.21.15.107:6789/0 conn(0x7fd4b00cedf0 legacy=0x7fd4b00b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.505+0000 7fd4b5000640 1 -- 172.21.15.49:0/276017842 >> v1:172.21.15.49:6789/0 conn(0x7fd4b00b3fe0 legacy=0x7fd4b015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.505+0000 7fd4b5000640 1 -- 172.21.15.49:0/276017842 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b00b4900 con 0x7fd4b01461f0 2024-04-15T15:20:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.505+0000 7fd4af400640 1 -- 172.21.15.49:0/276017842 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1199758331 0 0) 0x7fd4b00b4be0 con 0x7fd4b00b3fe0 2024-04-15T15:20:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.506+0000 7fd4b5000640 1 --2- 172.21.15.49:0/276017842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 0x7fd4b00ce8b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40097a0 tx=0x7fd4a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cedfa2c23cf75b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.506+0000 7fd4af400640 1 -- 172.21.15.49:0/276017842 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4035020 con 0x7fd4b01461f0 2024-04-15T15:20:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.507+0000 7fd4af400640 1 -- 172.21.15.49:0/276017842 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a4031830 con 0x7fd4b01461f0 2024-04-15T15:20:09.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.507+0000 7fd4af400640 1 -- 172.21.15.49:0/276017842 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4031c00 con 0x7fd4b01461f0 2024-04-15T15:20:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.507+0000 7fd4b7a00640 1 -- 172.21.15.49:0/276017842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 msgr2=0x7fd4b00ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.507+0000 7fd4b7a00640 1 --2- 172.21.15.49:0/276017842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 0x7fd4b00ce8b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40097a0 tx=0x7fd4a402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.507+0000 7fd4b7a00640 1 -- 172.21.15.49:0/276017842 shutdown_connections 2024-04-15T15:20:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.507+0000 7fd4b7a00640 1 --2- 172.21.15.49:0/276017842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 0x7fd4b00ce8b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.507+0000 7fd4b7a00640 1 -- 172.21.15.49:0/276017842 >> 172.21.15.49:0/276017842 conn(0x7fd4b00b27b0 msgr2=0x7fd4b00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.508+0000 7fd4b7a00640 1 -- 172.21.15.49:0/276017842 shutdown_connections 2024-04-15T15:20:09.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.508+0000 7fd4b7a00640 1 -- 172.21.15.49:0/276017842 wait complete. 2024-04-15T15:20:09.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.509+0000 7fd4b7a00640 1 Processor -- start 2024-04-15T15:20:09.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.510+0000 7fd4b7a00640 1 -- start start 2024-04-15T15:20:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4b7a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4b00b3fe0 0x7fd4b01e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4b7a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4b00cedf0 0x7fd4b01f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4b7a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 0x7fd4b00c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4b7a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b015d290 con 0x7fd4b01461f0 2024-04-15T15:20:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4b7a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b01465d0 con 0x7fd4b00b3fe0 2024-04-15T15:20:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4b7a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b015dec0 con 0x7fd4b00cedf0 2024-04-15T15:20:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4b5a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 0x7fd4b00c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4b5a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 0x7fd4b00c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42968/0 (socket says 172.21.15.49:42968) 2024-04-15T15:20:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4b5a00640 1 -- 172.21.15.49:0/1889645076 learned_addr learned my addr 172.21.15.49:0/1889645076 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4b5000640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4b00b3fe0 0x7fd4b01e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.511+0000 7fd4afe00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4b00cedf0 0x7fd4b01f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4b5a00640 1 -- 172.21.15.49:0/1889645076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4b00cedf0 msgr2=0x7fd4b01f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4b5a00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4b00cedf0 0x7fd4b01f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4b5a00640 1 -- 172.21.15.49:0/1889645076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4b00b3fe0 msgr2=0x7fd4b01e5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4b5a00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4b00b3fe0 0x7fd4b01e5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4b5a00640 1 -- 172.21.15.49:0/1889645076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b01e62b0 con 0x7fd4b01461f0 2024-04-15T15:20:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4b5000640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4b00b3fe0 0x7fd4b01e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4b5a00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 0x7fd4b00c5ae0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800c7a0 tx=0x7fd4a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4acc00640 1 -- 172.21.15.49:0/1889645076 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8018070 con 0x7fd4b01461f0 2024-04-15T15:20:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4acc00640 1 -- 172.21.15.49:0/1889645076 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a800a640 con 0x7fd4b01461f0 2024-04-15T15:20:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4acc00640 1 -- 172.21.15.49:0/1889645076 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800aab0 con 0x7fd4b01461f0 2024-04-15T15:20:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4b7a00640 1 -- 172.21.15.49:0/1889645076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b01e68a0 con 0x7fd4b01461f0 2024-04-15T15:20:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.512+0000 7fd4b7a00640 1 -- 172.21.15.49:0/1889645076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b01e6e60 con 0x7fd4b01461f0 2024-04-15T15:20:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.514+0000 7fd4b7a00640 1 -- 172.21.15.49:0/1889645076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b00d0000 con 0x7fd4b01461f0 2024-04-15T15:20:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.515+0000 7fd4acc00640 1 -- 172.21.15.49:0/1889645076 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4a801b070 con 0x7fd4b01461f0 2024-04-15T15:20:09.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.516+0000 7fd4acc00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd49006c8f0 0x7fd49006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:09.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.516+0000 7fd4acc00640 1 -- 172.21.15.49:0/1889645076 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd4a808b0b0 con 0x7fd4b01461f0 2024-04-15T15:20:09.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.517+0000 7fd4b5000640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd49006c8f0 0x7fd49006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:09.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.518+0000 7fd4b5000640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd49006c8f0 0x7fd49006edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4036000 tx=0x7fd4a4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.521+0000 7fd4acc00640 1 -- 172.21.15.49:0/1889645076 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4b00d0000 con 0x7fd4b01461f0 2024-04-15T15:20:09.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.773+0000 7fd4b7a00640 1 -- 172.21.15.49:0/1889645076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd4b0068c80 con 0x7fd4b01461f0 2024-04-15T15:20:09.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.774+0000 7fd4acc00640 1 -- 172.21.15.49:0/1889645076 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fd4a8059570 con 0x7fd4b01461f0 2024-04-15T15:20:09.776 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 -- 172.21.15.49:0/1889645076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd49006c8f0 msgr2=0x7fd49006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd49006c8f0 0x7fd49006edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4036000 tx=0x7fd4a4008030 comp rx=0 tx=0).stop 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 -- 172.21.15.49:0/1889645076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 msgr2=0x7fd4b00c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 0x7fd4b00c5ae0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800c7a0 tx=0x7fd4a800cc60 comp rx=0 tx=0).stop 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 -- 172.21.15.49:0/1889645076 shutdown_connections 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd49006c8f0 0x7fd49006edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4b00cedf0 0x7fd4b01f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b01461f0 0x7fd4b00c5ae0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 --2- 172.21.15.49:0/1889645076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4b00b3fe0 0x7fd4b01e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:09.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 -- 172.21.15.49:0/1889645076 >> 172.21.15.49:0/1889645076 conn(0x7fd4b00b27b0 msgr2=0x7fd4b015eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:09.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 -- 172.21.15.49:0/1889645076 shutdown_connections 2024-04-15T15:20:09.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:09.777+0000 7fd4b7a00640 1 -- 172.21.15.49:0/1889645076 wait complete. 2024-04-15T15:20:09.802 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:20:09.803 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:09.803 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-04-15T15:20:09.803 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-15T15:20:09.803 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:09.803 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:20:09.803 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-15T15:20:09.803 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:09.803 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:09.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:09.803 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-04-15T15:20:09.860 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-04-15T15:20:09.861 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42512sec preferred_lft 42512sec 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 82:38:d9:8f:e0:b7 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::26:49ff:feaf:7d0d/64 scope link 2024-04-15T15:20:09.862 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:09.863 INFO:teuthology.orchestra.run.smithi107.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:09.863 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether fe:39:d9:9f:a9:b1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:09.863 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::fc39:d9ff:fe9f:a9b1/64 scope link 2024-04-15T15:20:09.863 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:09.863 INFO:teuthology.orchestra.run.smithi107.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:09.863 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 82:38:d9:8f:e0:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:09.863 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::8038:d9ff:fe8f:e0b7/64 scope link 2024-04-15T15:20:09.863 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:09.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:09.864 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:09.952 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-15T15:20:09.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:09.956 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-04-15T15:20:10.006 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-04-15T15:20:10.006 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:10.006 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:20:10.006 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:20:10.007 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:10.007 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:10.007 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:20:10.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:20:10.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:20:10.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:20:10.007 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:20:10.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:10.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:10.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:10.242 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-15T15:20:10.292 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:20:10.292 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:20:10.292 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:10.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:10.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-04-15T15:20:10.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:10.386 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:10.440 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:10.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:10.440 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:10.491 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:20:10.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:10.491 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:20:10.628 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:20:10.628 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:20:10.628 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:20:10.650 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:20:10.650 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:20:10.650 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:20:10.650 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:10.650 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:10.677 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:20:10.677 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:20:10.677 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:20:10.677 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:10.677 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:10.677 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790" 2024-04-15T15:20:10.677 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790:/ 2024-04-15T15:20:10.677 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c" will pass to kernel 2024-04-15T15:20:10.680 INFO:teuthology.orchestra.run.smithi107.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-15T15:20:10.680 INFO:teuthology.orchestra.run.smithi107.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:20:10.680 INFO:teuthology.orchestra.run.smithi107.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:20:10.680 INFO:teuthology.orchestra.run.smithi107.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:20:10.680 INFO:teuthology.orchestra.run.smithi107.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:20:10.684 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:20:10.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:10.685 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:10.766 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:20:10.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:20:10.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:10.887 INFO:teuthology.orchestra.run.smithi107.stdout:client5256 2024-04-15T15:20:10.888 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:20:10.888 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:10.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds debug_mds 25 2024-04-15T15:20:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.178+0000 7f6f11c00640 1 Processor -- start 2024-04-15T15:20:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.178+0000 7f6f11c00640 1 -- start start 2024-04-15T15:20:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.179+0000 7f6f11c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6f0c1461c0 0x7f6f0c071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.179+0000 7f6f11c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6f0c072420 0x7f6f0c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.179+0000 7f6f11c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6f0c0ce9e0 0x7f6f0c160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.179+0000 7f6f11c00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0c0cf1d0 con 0x7f6f0c0ce9e0 2024-04-15T15:20:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.179+0000 7f6f11c00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0c072880 con 0x7f6f0c1461c0 2024-04-15T15:20:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.179+0000 7f6f11c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c0cd080 con 0x7f6f0c072420 2024-04-15T15:20:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.180+0000 7f6f11200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6f0c0ce9e0 0x7f6f0c160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.180+0000 7f6f11200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6f0c0ce9e0 0x7f6f0c160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42980/0 (socket says 172.21.15.49:42980) 2024-04-15T15:20:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.180+0000 7f6f11200640 1 -- 172.21.15.49:0/2462552935 learned_addr learned my addr 172.21.15.49:0/2462552935 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.180+0000 7f6f03e00640 1 --2- 172.21.15.49:0/2462552935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6f0c072420 0x7f6f0c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.180+0000 7f6f0be00640 1 --2- 172.21.15.49:0/2462552935 >> v2:172.21.15.107:3300/0 conn(0x7f6f0c1461c0 0x7f6f0c071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.180+0000 7f6f11200640 1 -- 172.21.15.49:0/2462552935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6f0c072420 msgr2=0x7f6f0c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.180+0000 7f6f11200640 1 --2- 172.21.15.49:0/2462552935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6f0c072420 0x7f6f0c0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.180+0000 7f6f11200640 1 -- 172.21.15.49:0/2462552935 >> v2:172.21.15.107:3300/0 conn(0x7f6f0c1461c0 msgr2=0x7f6f0c071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.180+0000 7f6f11200640 1 --2- 172.21.15.49:0/2462552935 >> v2:172.21.15.107:3300/0 conn(0x7f6f0c1461c0 0x7f6f0c071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.180+0000 7f6f11200640 1 -- 172.21.15.49:0/2462552935 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c0cd2d0 con 0x7f6f0c0ce9e0 2024-04-15T15:20:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.181+0000 7f6f11200640 1 --2- 172.21.15.49:0/2462552935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c0ce9e0 0x7f6f0c160200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0d05c0 tx=0x7f6f0402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d04c30f73320275 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.182+0000 7f6f0b400640 1 -- 172.21.15.49:0/2462552935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0402ca80 con 0x7f6f0c0ce9e0 2024-04-15T15:20:11.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.182+0000 7f6f0b400640 1 -- 172.21.15.49:0/2462552935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0402cc20 con 0x7f6f0c0ce9e0 2024-04-15T15:20:11.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.182+0000 7f6f0b400640 1 -- 172.21.15.49:0/2462552935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040343f0 con 0x7f6f0c0ce9e0 2024-04-15T15:20:11.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.182+0000 7f6f11c00640 1 -- 172.21.15.49:0/2462552935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c0ce9e0 msgr2=0x7f6f0c160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.182+0000 7f6f11c00640 1 --2- 172.21.15.49:0/2462552935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c0ce9e0 0x7f6f0c160200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0d05c0 tx=0x7f6f0402bee0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.183+0000 7f6f11c00640 1 -- 172.21.15.49:0/2462552935 shutdown_connections 2024-04-15T15:20:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.183+0000 7f6f11c00640 1 --2- 172.21.15.49:0/2462552935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c0ce9e0 0x7f6f0c160200 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.183+0000 7f6f11c00640 1 --2- 172.21.15.49:0/2462552935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6f0c072420 0x7f6f0c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.183+0000 7f6f11c00640 1 --2- 172.21.15.49:0/2462552935 >> v2:172.21.15.107:3300/0 conn(0x7f6f0c1461c0 0x7f6f0c071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.183+0000 7f6f11c00640 1 -- 172.21.15.49:0/2462552935 >> 172.21.15.49:0/2462552935 conn(0x7f6f0c06b930 msgr2=0x7f6f0c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.183+0000 7f6f11c00640 1 -- 172.21.15.49:0/2462552935 shutdown_connections 2024-04-15T15:20:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.183+0000 7f6f11c00640 1 -- 172.21.15.49:0/2462552935 wait complete. 2024-04-15T15:20:11.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.184+0000 7f6f11c00640 1 Processor -- start 2024-04-15T15:20:11.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.185+0000 7f6f11c00640 1 -- start start 2024-04-15T15:20:11.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.185+0000 7f6f11c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c072420 0x7f6f0c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.185+0000 7f6f11c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ce9e0 0x7f6f0c15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.185+0000 7f6f11c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6f0c1461c0 0x7f6f0c154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.185+0000 7f6f11c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c0cdb70 con 0x7f6f0c072420 2024-04-15T15:20:11.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.185+0000 7f6f11c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c0cd890 con 0x7f6f0c0ce9e0 2024-04-15T15:20:11.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.185+0000 7f6f11c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c0cee70 con 0x7f6f0c1461c0 2024-04-15T15:20:11.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f0be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c072420 0x7f6f0c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f0be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c072420 0x7f6f0c1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42994/0 (socket says 172.21.15.49:42994) 2024-04-15T15:20:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f0be00640 1 -- 172.21.15.49:0/3352954641 learned_addr learned my addr 172.21.15.49:0/3352954641 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f11200640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6f0c1461c0 0x7f6f0c154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f03e00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ce9e0 0x7f6f0c15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f0be00640 1 -- 172.21.15.49:0/3352954641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6f0c1461c0 msgr2=0x7f6f0c154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f0be00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6f0c1461c0 0x7f6f0c154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f0be00640 1 -- 172.21.15.49:0/3352954641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ce9e0 msgr2=0x7f6f0c15ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f0be00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ce9e0 0x7f6f0c15ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f0be00640 1 -- 172.21.15.49:0/3352954641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c157e80 con 0x7f6f0c072420 2024-04-15T15:20:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.186+0000 7f6f03e00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ce9e0 0x7f6f0c15ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:11.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.187+0000 7f6f0be00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c072420 0x7f6f0c1599b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00d5f0 tx=0x7f6efc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.187+0000 7f6f08c00640 1 -- 172.21.15.49:0/3352954641 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0125f0 con 0x7f6f0c072420 2024-04-15T15:20:11.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.187+0000 7f6f08c00640 1 -- 172.21.15.49:0/3352954641 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6efc019070 con 0x7f6f0c072420 2024-04-15T15:20:11.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.187+0000 7f6f08c00640 1 -- 172.21.15.49:0/3352954641 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0129d0 con 0x7f6f0c072420 2024-04-15T15:20:11.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.187+0000 7f6f11c00640 1 -- 172.21.15.49:0/3352954641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c158160 con 0x7f6f0c072420 2024-04-15T15:20:11.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.187+0000 7f6f11c00640 1 -- 172.21.15.49:0/3352954641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c158720 con 0x7f6f0c072420 2024-04-15T15:20:11.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.188+0000 7f6f11c00640 1 -- 172.21.15.49:0/3352954641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0c158b60 con 0x7f6f0c072420 2024-04-15T15:20:11.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.189+0000 7f6f08c00640 1 -- 172.21.15.49:0/3352954641 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6efc012b70 con 0x7f6f0c072420 2024-04-15T15:20:11.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.190+0000 7f6f08c00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6ee806c8f0 0x7f6ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.190+0000 7f6f08c00640 1 -- 172.21.15.49:0/3352954641 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6efc08be30 con 0x7f6f0c072420 2024-04-15T15:20:11.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.191+0000 7f6f03e00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6ee806c8f0 0x7f6ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.191+0000 7f6f03e00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6ee806c8f0 0x7f6ee806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c167560 tx=0x7f6ef4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.195+0000 7f6f08c00640 1 -- 172.21.15.49:0/3352954641 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6efc05a2f0 con 0x7f6f0c072420 2024-04-15T15:20:11.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.401+0000 7f6f11c00640 1 -- 172.21.15.49:0/3352954641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f6f0c0755c0 con 0x7f6f0c072420 2024-04-15T15:20:11.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.406+0000 7f6f08c00640 1 -- 172.21.15.49:0/3352954641 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v11)=0 v11) ==== 108+0+0 (secure 0 0 0) 0x7f6efc059bf0 con 0x7f6f0c072420 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.410+0000 7f6f11c00640 1 -- 172.21.15.49:0/3352954641 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6ee806c8f0 msgr2=0x7f6ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.410+0000 7f6f11c00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6ee806c8f0 0x7f6ee806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c167560 tx=0x7f6ef4008040 comp rx=0 tx=0).stop 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.410+0000 7f6f11c00640 1 -- 172.21.15.49:0/3352954641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c072420 msgr2=0x7f6f0c1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.410+0000 7f6f11c00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c072420 0x7f6f0c1599b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00d5f0 tx=0x7f6efc00dab0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.411+0000 7f6f11c00640 1 -- 172.21.15.49:0/3352954641 shutdown_connections 2024-04-15T15:20:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.411+0000 7f6f11c00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6ee806c8f0 0x7f6ee806edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.411+0000 7f6f11c00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6f0c1461c0 0x7f6f0c154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.411+0000 7f6f11c00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ce9e0 0x7f6f0c15ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.411+0000 7f6f11c00640 1 --2- 172.21.15.49:0/3352954641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0c072420 0x7f6f0c1599b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.411+0000 7f6f11c00640 1 -- 172.21.15.49:0/3352954641 >> 172.21.15.49:0/3352954641 conn(0x7f6f0c06b930 msgr2=0x7f6f0c1363a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:11.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.411+0000 7f6f11c00640 1 -- 172.21.15.49:0/3352954641 shutdown_connections 2024-04-15T15:20:11.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.411+0000 7f6f11c00640 1 -- 172.21.15.49:0/3352954641 wait complete. 2024-04-15T15:20:11.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_quiesce_splitauth true 2024-04-15T15:20:11.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.667+0000 7fd90fe00640 1 Processor -- start 2024-04-15T15:20:11.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.667+0000 7fd90fe00640 1 -- start start 2024-04-15T15:20:11.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.668+0000 7fd90fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd91015e9a0 0x7fd910162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90fe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9100ba3e0 0x7fd9100cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90fe00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd9100cfa30 0x7fd9100b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90fe00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd910163450 con 0x7fd9100ba3e0 2024-04-15T15:20:11.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90fe00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fd910162e90 con 0x7fd9100cfa30 2024-04-15T15:20:11.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90fe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd910163170 con 0x7fd91015e9a0 2024-04-15T15:20:11.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90e000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9100ba3e0 0x7fd9100cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90e000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9100ba3e0 0x7fd9100cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43004/0 (socket says 172.21.15.49:43004) 2024-04-15T15:20:11.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90e000640 1 -- 172.21.15.49:0/4015667761 learned_addr learned my addr 172.21.15.49:0/4015667761 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:11.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90ea00640 1 --2- 172.21.15.49:0/4015667761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd91015e9a0 0x7fd910162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90f400640 1 --2- 172.21.15.49:0/4015667761 >> v2:172.21.15.107:3300/0 conn(0x7fd9100cfa30 0x7fd9100b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90e000640 1 -- 172.21.15.49:0/4015667761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd91015e9a0 msgr2=0x7fd910162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:11.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90e000640 1 --2- 172.21.15.49:0/4015667761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd91015e9a0 0x7fd910162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90e000640 1 -- 172.21.15.49:0/4015667761 >> v2:172.21.15.107:3300/0 conn(0x7fd9100cfa30 msgr2=0x7fd9100b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:11.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90e000640 1 --2- 172.21.15.49:0/4015667761 >> v2:172.21.15.107:3300/0 conn(0x7fd9100cfa30 0x7fd9100b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.669+0000 7fd90e000640 1 -- 172.21.15.49:0/4015667761 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9100d2920 con 0x7fd9100ba3e0 2024-04-15T15:20:11.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.670+0000 7fd90ea00640 1 --2- 172.21.15.49:0/4015667761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd91015e9a0 0x7fd910162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:11.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.670+0000 7fd90f400640 1 --2- 172.21.15.49:0/4015667761 >> v2:172.21.15.107:3300/0 conn(0x7fd9100cfa30 0x7fd9100b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:11.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.671+0000 7fd90e000640 1 --2- 172.21.15.49:0/4015667761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9100ba3e0 0x7fd9100cf4f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd9100b7690 tx=0x7fd8fc02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca586b633c8613a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.671+0000 7fd90d600640 1 -- 172.21.15.49:0/4015667761 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc034070 con 0x7fd9100ba3e0 2024-04-15T15:20:11.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.671+0000 7fd90d600640 1 -- 172.21.15.49:0/4015667761 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8fc030850 con 0x7fd9100ba3e0 2024-04-15T15:20:11.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.671+0000 7fd90d600640 1 -- 172.21.15.49:0/4015667761 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc030c20 con 0x7fd9100ba3e0 2024-04-15T15:20:11.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.672+0000 7fd90fe00640 1 -- 172.21.15.49:0/4015667761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9100ba3e0 msgr2=0x7fd9100cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.672+0000 7fd90fe00640 1 --2- 172.21.15.49:0/4015667761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9100ba3e0 0x7fd9100cf4f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd9100b7690 tx=0x7fd8fc02b990 comp rx=0 tx=0).stop 2024-04-15T15:20:11.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.672+0000 7fd90fe00640 1 -- 172.21.15.49:0/4015667761 shutdown_connections 2024-04-15T15:20:11.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.672+0000 7fd90fe00640 1 --2- 172.21.15.49:0/4015667761 >> v2:172.21.15.107:3300/0 conn(0x7fd9100cfa30 0x7fd9100b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.672+0000 7fd90fe00640 1 --2- 172.21.15.49:0/4015667761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9100ba3e0 0x7fd9100cf4f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.672+0000 7fd90fe00640 1 --2- 172.21.15.49:0/4015667761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd91015e9a0 0x7fd910162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.672+0000 7fd90fe00640 1 -- 172.21.15.49:0/4015667761 >> 172.21.15.49:0/4015667761 conn(0x7fd91006ce40 msgr2=0x7fd91006d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:11.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.672+0000 7fd90fe00640 1 -- 172.21.15.49:0/4015667761 shutdown_connections 2024-04-15T15:20:11.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.672+0000 7fd90fe00640 1 -- 172.21.15.49:0/4015667761 wait complete. 2024-04-15T15:20:11.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.674+0000 7fd90fe00640 1 Processor -- start 2024-04-15T15:20:11.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.674+0000 7fd90fe00640 1 -- start start 2024-04-15T15:20:11.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.675+0000 7fd90fe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9100ba3e0 0x7fd9101e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.675+0000 7fd90fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9100cfa30 0x7fd9100caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.675+0000 7fd90fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91015e9a0 0x7fd910142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.675+0000 7fd90fe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd91015ed80 con 0x7fd91015e9a0 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.675+0000 7fd90fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd910162ad0 con 0x7fd9100cfa30 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.675+0000 7fd90fe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd9101e8ba0 con 0x7fd9100ba3e0 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.675+0000 7fd90f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91015e9a0 0x7fd910142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.675+0000 7fd90f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91015e9a0 0x7fd910142f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43014/0 (socket says 172.21.15.49:43014) 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.675+0000 7fd90f400640 1 -- 172.21.15.49:0/191932282 learned_addr learned my addr 172.21.15.49:0/191932282 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.676+0000 7fd90ea00640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9100ba3e0 0x7fd9101e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.676+0000 7fd90e000640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9100cfa30 0x7fd9100caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.676+0000 7fd90f400640 1 -- 172.21.15.49:0/191932282 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9100ba3e0 msgr2=0x7fd9101e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.676+0000 7fd90f400640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9100ba3e0 0x7fd9101e85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.676+0000 7fd90f400640 1 -- 172.21.15.49:0/191932282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9100cfa30 msgr2=0x7fd9100caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.676+0000 7fd90f400640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9100cfa30 0x7fd9100caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.676+0000 7fd90f400640 1 -- 172.21.15.49:0/191932282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9101e8e80 con 0x7fd91015e9a0 2024-04-15T15:20:11.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.676+0000 7fd90e000640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9100cfa30 0x7fd9100caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:11.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.676+0000 7fd90f400640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91015e9a0 0x7fd910142f40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd90800cd10 tx=0x7fd90800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.676+0000 7fd907400640 1 -- 172.21.15.49:0/191932282 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90800f680 con 0x7fd91015e9a0 2024-04-15T15:20:11.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.677+0000 7fd907400640 1 -- 172.21.15.49:0/191932282 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd90800f840 con 0x7fd91015e9a0 2024-04-15T15:20:11.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.677+0000 7fd907400640 1 -- 172.21.15.49:0/191932282 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90800fc30 con 0x7fd91015e9a0 2024-04-15T15:20:11.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.677+0000 7fd90fe00640 1 -- 172.21.15.49:0/191932282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9101e9160 con 0x7fd91015e9a0 2024-04-15T15:20:11.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.677+0000 7fd90fe00640 1 -- 172.21.15.49:0/191932282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9101e9720 con 0x7fd91015e9a0 2024-04-15T15:20:11.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.678+0000 7fd90fe00640 1 -- 172.21.15.49:0/191932282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9100cb060 con 0x7fd91015e9a0 2024-04-15T15:20:11.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.679+0000 7fd907400640 1 -- 172.21.15.49:0/191932282 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9080213f0 con 0x7fd91015e9a0 2024-04-15T15:20:11.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.680+0000 7fd907400640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd8f006c8f0 0x7fd8f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:11.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.680+0000 7fd907400640 1 -- 172.21.15.49:0/191932282 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd90808c920 con 0x7fd91015e9a0 2024-04-15T15:20:11.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.680+0000 7fd90ea00640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd8f006c8f0 0x7fd8f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:11.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.681+0000 7fd90ea00640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd8f006c8f0 0x7fd8f006edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd90000a990 tx=0x7fd900008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:11.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.686+0000 7fd907400640 1 -- 172.21.15.49:0/191932282 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd908014030 con 0x7fd91015e9a0 2024-04-15T15:20:11.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.889+0000 7fd90fe00640 1 -- 172.21.15.49:0/191932282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mds_cache_quiesce_splitauth}] v 0) -- 0x7fd910068ef0 con 0x7fd91015e9a0 2024-04-15T15:20:11.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.894+0000 7fd907400640 1 -- 172.21.15.49:0/191932282 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_quiesce_splitauth}]=0 v12)=0 v12) ==== 128+0+0 (secure 0 0 0) 0x7fd90805a660 con 0x7fd91015e9a0 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 -- 172.21.15.49:0/191932282 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd8f006c8f0 msgr2=0x7fd8f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd8f006c8f0 0x7fd8f006edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd90000a990 tx=0x7fd900008040 comp rx=0 tx=0).stop 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 -- 172.21.15.49:0/191932282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91015e9a0 msgr2=0x7fd910142f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91015e9a0 0x7fd910142f40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd90800cd10 tx=0x7fd90800a650 comp rx=0 tx=0).stop 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 -- 172.21.15.49:0/191932282 shutdown_connections 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd8f006c8f0 0x7fd8f006edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91015e9a0 0x7fd910142f40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9100cfa30 0x7fd9100caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 --2- 172.21.15.49:0/191932282 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9100ba3e0 0x7fd9101e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 -- 172.21.15.49:0/191932282 >> 172.21.15.49:0/191932282 conn(0x7fd91006ce40 msgr2=0x7fd910159400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:11.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.897+0000 7fd90fe00640 1 -- 172.21.15.49:0/191932282 shutdown_connections 2024-04-15T15:20:11.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:11.898+0000 7fd90fe00640 1 -- 172.21.15.49:0/191932282 wait complete. 2024-04-15T15:20:11.920 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol_quiesce --mode=777 2024-04-15T15:20:12.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.148+0000 7f1fb3200640 1 Processor -- start 2024-04-15T15:20:12.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.149+0000 7f1fb3200640 1 -- start start 2024-04-15T15:20:12.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.150+0000 7f1fb3200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb415eac0 0x7f1fb40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.150+0000 7f1fb3200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1fb40b6cf0 0x7f1fb40ba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.150+0000 7f1fb3200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb40d2f10 con 0x7f1fb40d2ad0 2024-04-15T15:20:12.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.150+0000 7f1fb3200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb415eea0 con 0x7f1fb40b6cf0 2024-04-15T15:20:12.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.150+0000 7f1fb3200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb40d3190 con 0x7f1fb415eac0 2024-04-15T15:20:12.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.151+0000 7f1fa9400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1fb40d2ad0 0x7f1fb40b65e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:40414/0 (socket says 172.21.15.49:40414) 2024-04-15T15:20:12.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.151+0000 7f1fa9400640 1 -- 172.21.15.49:0/3656769479 learned_addr learned my addr 172.21.15.49:0/3656769479 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:12.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.151+0000 7f1fb2800640 1 --2- 172.21.15.49:0/3656769479 >> v2:172.21.15.107:3300/0 conn(0x7f1fb40b6cf0 0x7f1fb40ba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.151+0000 7f1fb1e00640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb415eac0 0x7f1fb40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.151+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453945855 0 0) 0x7f1f9c001110 con 0x7f1fb40d2ad0 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.151+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f98003520 con 0x7f1fb40d2ad0 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3477726669 0 0) 0x7f1f98003520 con 0x7f1fb40d2ad0 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb415eac0 msgr2=0x7f1fb40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb415eac0 0x7f1fb40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 >> v2:172.21.15.107:3300/0 conn(0x7f1fb40b6cf0 msgr2=0x7f1fb40ba2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 --2- 172.21.15.49:0/3656769479 >> v2:172.21.15.107:3300/0 conn(0x7f1fb40b6cf0 0x7f1fb40ba2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb40d3470 con 0x7f1fb40d2ad0 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f1f9c001840 con 0x7f1fb40d2ad0 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 >> v1:172.21.15.49:6789/0 conn(0x7f1fb40d2ad0 legacy=0x7f1fb40b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb415eac0 0x7f1f980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 0x7f1f98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f98004d40 0x7f1f980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb415eea0 con 0x7f1f98003f30 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb40d3190 con 0x7f1f98004d40 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9800b9f0 con 0x7f1fb415eac0 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.152+0000 7f1fa9400640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 0x7f1f98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.153+0000 7f1fb1e00640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb415eac0 0x7f1f980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.153+0000 7f1fa9400640 1 -- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb415eac0 msgr2=0x7f1f980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.153+0000 7f1fa9400640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb415eac0 0x7f1f980039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.153+0000 7f1fb2800640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f98004d40 0x7f1f980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.153+0000 7f1fa9400640 1 -- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f98004d40 msgr2=0x7f1f980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.153+0000 7f1fa9400640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f98004d40 0x7f1f980051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.153+0000 7f1fa9400640 1 -- 172.21.15.49:0/3656769479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f98003520 con 0x7f1f98003f30 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.154+0000 7f1fa9400640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 0x7f1f98004800 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c3e0 tx=0x7f1f98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c66ad73807205bc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.154+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00c0e0 con 0x7f1f98003f30 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.154+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9c030070 con 0x7f1f98003f30 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.154+0000 7f1fb1400640 1 -- 172.21.15.49:0/3656769479 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0058d0 con 0x7f1f98003f30 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.155+0000 7f1fb3200640 1 -- 172.21.15.49:0/3656769479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 msgr2=0x7f1f98004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.155+0000 7f1fb3200640 1 --2- 172.21.15.49:0/3656769479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 0x7f1f98004800 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c3e0 tx=0x7f1f98005830 comp rx=0 tx=0).stop 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.155+0000 7f1fb2800640 1 -- 172.21.15.49:0/3656769479 reap_dead start 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.155+0000 7f1fb3200640 1 -- 172.21.15.49:0/3656769479 shutdown_connections 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.155+0000 7f1fb3200640 1 -- 172.21.15.49:0/3656769479 >> 172.21.15.49:0/3656769479 conn(0x7f1fb40d1650 msgr2=0x7f1fb40d1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.155+0000 7f1fb3200640 1 -- 172.21.15.49:0/3656769479 shutdown_connections 2024-04-15T15:20:12.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.155+0000 7f1fb3200640 1 -- 172.21.15.49:0/3656769479 wait complete. 2024-04-15T15:20:12.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.157+0000 7f1fb3200640 1 Processor -- start 2024-04-15T15:20:12.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.157+0000 7f1fb3200640 1 -- start start 2024-04-15T15:20:12.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb3200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 0x7f1fb40ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb41e97f0 0x7f1fb41e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb3200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb41e5b10 0x7f1fb41e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb3200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb40d3d10 con 0x7f1f98003f30 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb40d3a30 con 0x7f1fb41e97f0 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb3200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb40d4230 con 0x7f1fb41e5b10 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 0x7f1fb40ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb1e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 0x7f1fb40ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43020/0 (socket says 172.21.15.49:43020) 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb1e00640 1 -- 172.21.15.49:0/3977773184 learned_addr learned my addr 172.21.15.49:0/3977773184 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb1e00640 1 -- 172.21.15.49:0/3977773184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb41e5b10 msgr2=0x7f1fb41e5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb1e00640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb41e5b10 0x7f1fb41e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fa9400640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb41e97f0 0x7f1fb41e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb1e00640 1 -- 172.21.15.49:0/3977773184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb41e97f0 msgr2=0x7f1fb41e9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb1e00640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb41e97f0 0x7f1fb41e9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.158+0000 7f1fb1e00640 1 -- 172.21.15.49:0/3977773184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb41ea1b0 con 0x7f1f98003f30 2024-04-15T15:20:12.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.159+0000 7f1fb1e00640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 0x7f1fb40ca610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4010840 tx=0x7f1fa4010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:12.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.159+0000 7f1faaa00640 1 -- 172.21.15.49:0/3977773184 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400f040 con 0x7f1f98003f30 2024-04-15T15:20:12.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.159+0000 7f1faaa00640 1 -- 172.21.15.49:0/3977773184 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa400eb60 con 0x7f1f98003f30 2024-04-15T15:20:12.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.159+0000 7f1faaa00640 1 -- 172.21.15.49:0/3977773184 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa401c400 con 0x7f1f98003f30 2024-04-15T15:20:12.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.159+0000 7f1fb3200640 1 -- 172.21.15.49:0/3977773184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb4149990 con 0x7f1f98003f30 2024-04-15T15:20:12.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.159+0000 7f1fb3200640 1 -- 172.21.15.49:0/3977773184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb4149c30 con 0x7f1f98003f30 2024-04-15T15:20:12.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.160+0000 7f1fb3200640 1 -- 172.21.15.49:0/3977773184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb4068be0 con 0x7f1f98003f30 2024-04-15T15:20:12.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.161+0000 7f1faaa00640 1 -- 172.21.15.49:0/3977773184 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fa401c5a0 con 0x7f1f98003f30 2024-04-15T15:20:12.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.162+0000 7f1faaa00640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f9006c920 0x7f1f9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.162+0000 7f1faaa00640 1 -- 172.21.15.49:0/3977773184 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1fa408c090 con 0x7f1f98003f30 2024-04-15T15:20:12.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.163+0000 7f1fa9400640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f9006c920 0x7f1f9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.163+0000 7f1fa9400640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f9006c920 0x7f1f9006ede0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c3b0 tx=0x7f1f9c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:12.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.167+0000 7f1faaa00640 1 -- 172.21.15.49:0/3977773184 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fa4018030 con 0x7f1f98003f30 2024-04-15T15:20:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.403+0000 7f1fb3200640 1 -- 172.21.15.49:0/3977773184 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol_quiesce", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f1fb40cab50 con 0x7f1f9006c920 2024-04-15T15:20:12.419 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:20:12.418+0000 7f36bea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:20:12.419 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:20:12.418+0000 7f36bea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:20:12.420 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:20:12.418+0000 7f36bea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:20:12.420 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:20:12.418+0000 7f36bea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:20:12.420 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-15T15:20:12.418+0000 7f36bea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:20:12.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.466+0000 7f1faaa00640 1 -- 172.21.15.49:0/3977773184 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1fb40cab50 con 0x7f1f9006c920 2024-04-15T15:20:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.469+0000 7f1fb3200640 1 -- 172.21.15.49:0/3977773184 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f9006c920 msgr2=0x7f1f9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.469+0000 7f1fb3200640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f9006c920 0x7f1f9006ede0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c3b0 tx=0x7f1f9c007000 comp rx=0 tx=0).stop 2024-04-15T15:20:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.469+0000 7f1fb3200640 1 -- 172.21.15.49:0/3977773184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 msgr2=0x7f1fb40ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.469+0000 7f1fb3200640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 0x7f1fb40ca610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4010840 tx=0x7f1fa4010d00 comp rx=0 tx=0).stop 2024-04-15T15:20:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.469+0000 7f1fb3200640 1 -- 172.21.15.49:0/3977773184 shutdown_connections 2024-04-15T15:20:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.469+0000 7f1fb3200640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f1f9006c920 0x7f1f9006ede0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.469+0000 7f1fb3200640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1fb41e5b10 0x7f1fb41e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.469+0000 7f1fb3200640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb41e97f0 0x7f1fb41e9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.469+0000 7f1fb3200640 1 --2- 172.21.15.49:0/3977773184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98003f30 0x7f1fb40ca610 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.469+0000 7f1fb3200640 1 -- 172.21.15.49:0/3977773184 >> 172.21.15.49:0/3977773184 conn(0x7f1fb40d1650 msgr2=0x7f1fb40cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.470+0000 7f1fb3200640 1 -- 172.21.15.49:0/3977773184 shutdown_connections 2024-04-15T15:20:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.470+0000 7f1fb3200640 1 -- 172.21.15.49:0/3977773184 wait complete. 2024-04-15T15:20:12.492 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_quiesce 2024-04-15T15:20:12.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.712+0000 7f4569600640 1 Processor -- start 2024-04-15T15:20:12.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.712+0000 7f4569600640 1 -- start start 2024-04-15T15:20:12.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.714+0000 7f4569600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4564147c90 0x7f456406f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.714+0000 7f4569600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f456406f750 0x7f45641626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.714+0000 7f4569600640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45640aebd0 con 0x7f45641478b0 2024-04-15T15:20:12.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.714+0000 7f4569600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f456406fbb0 con 0x7f456406f750 2024-04-15T15:20:12.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.714+0000 7f4569600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f45640aee50 con 0x7f4564147c90 2024-04-15T15:20:12.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.714+0000 7f4563e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f45641478b0 0x7f45640afbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:40418/0 (socket says 172.21.15.49:40418) 2024-04-15T15:20:12.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.714+0000 7f4563e00640 1 -- 172.21.15.49:0/1899597718 learned_addr learned my addr 172.21.15.49:0/1899597718 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:12.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.714+0000 7f4568c00640 1 --2- 172.21.15.49:0/1899597718 >> v2:172.21.15.49:3300/0 conn(0x7f456406f750 0x7f45641626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.715+0000 7f455be00640 1 --2- 172.21.15.49:0/1899597718 >> v2:172.21.15.107:3300/0 conn(0x7f4564147c90 0x7f456406f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.715+0000 7f4563400640 1 -- 172.21.15.49:0/1899597718 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603462777 0 0) 0x7f4554001110 con 0x7f45641478b0 2024-04-15T15:20:12.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.715+0000 7f4563400640 1 -- 172.21.15.49:0/1899597718 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4548003520 con 0x7f45641478b0 2024-04-15T15:20:12.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.715+0000 7f4568c00640 1 -- 172.21.15.49:0/1899597718 >> v2:172.21.15.107:3300/0 conn(0x7f4564147c90 msgr2=0x7f456406f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:12.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.715+0000 7f4568c00640 1 --2- 172.21.15.49:0/1899597718 >> v2:172.21.15.107:3300/0 conn(0x7f4564147c90 0x7f456406f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.715+0000 7f4568c00640 1 -- 172.21.15.49:0/1899597718 >> v1:172.21.15.49:6789/0 conn(0x7f45641478b0 legacy=0x7f45640afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.715+0000 7f4568c00640 1 -- 172.21.15.49:0/1899597718 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45640af110 con 0x7f456406f750 2024-04-15T15:20:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.716+0000 7f4568c00640 1 --2- 172.21.15.49:0/1899597718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f456406f750 0x7f45641626c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f45640d0500 tx=0x7f455c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec15dda96f009c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.716+0000 7f4563400640 1 -- 172.21.15.49:0/1899597718 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c039070 con 0x7f456406f750 2024-04-15T15:20:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.716+0000 7f4563400640 1 -- 172.21.15.49:0/1899597718 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c02cc80 con 0x7f456406f750 2024-04-15T15:20:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.716+0000 7f4563400640 1 -- 172.21.15.49:0/1899597718 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c034660 con 0x7f456406f750 2024-04-15T15:20:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.717+0000 7f4569600640 1 -- 172.21.15.49:0/1899597718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f456406f750 msgr2=0x7f45641626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.717+0000 7f4569600640 1 --2- 172.21.15.49:0/1899597718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f456406f750 0x7f45641626c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f45640d0500 tx=0x7f455c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.717+0000 7f4569600640 1 -- 172.21.15.49:0/1899597718 shutdown_connections 2024-04-15T15:20:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.717+0000 7f4569600640 1 --2- 172.21.15.49:0/1899597718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f456406f750 0x7f45641626c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.717+0000 7f4569600640 1 --2- 172.21.15.49:0/1899597718 >> v2:172.21.15.107:3300/0 conn(0x7f4564147c90 0x7f456406f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.717+0000 7f4569600640 1 -- 172.21.15.49:0/1899597718 >> 172.21.15.49:0/1899597718 conn(0x7f456406b930 msgr2=0x7f456406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.717+0000 7f4569600640 1 -- 172.21.15.49:0/1899597718 shutdown_connections 2024-04-15T15:20:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.717+0000 7f4569600640 1 -- 172.21.15.49:0/1899597718 wait complete. 2024-04-15T15:20:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.719+0000 7f4569600640 1 Processor -- start 2024-04-15T15:20:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.719+0000 7f4569600640 1 -- start start 2024-04-15T15:20:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.720+0000 7f4569600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456406f750 0x7f456415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.720+0000 7f4569600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45641478b0 0x7f45640b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.720+0000 7f4569600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4564147c90 0x7f45640b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.720+0000 7f4569600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4564162c00 con 0x7f45641478b0 2024-04-15T15:20:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.720+0000 7f4569600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45640af6d0 con 0x7f456406f750 2024-04-15T15:20:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.720+0000 7f4569600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f4564162d80 con 0x7f4564147c90 2024-04-15T15:20:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f455be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45641478b0 0x7f45640b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f4563e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456406f750 0x7f456415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f455be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45641478b0 0x7f45640b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43046/0 (socket says 172.21.15.49:43046) 2024-04-15T15:20:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f455be00640 1 -- 172.21.15.49:0/369078052 learned_addr learned my addr 172.21.15.49:0/369078052 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f4568c00640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4564147c90 0x7f45640b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f455be00640 1 -- 172.21.15.49:0/369078052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4564147c90 msgr2=0x7f45640b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f455be00640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4564147c90 0x7f45640b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f455be00640 1 -- 172.21.15.49:0/369078052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456406f750 msgr2=0x7f456415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f455be00640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456406f750 0x7f456415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f455be00640 1 -- 172.21.15.49:0/369078052 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456415e600 con 0x7f45641478b0 2024-04-15T15:20:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.721+0000 7f4568c00640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4564147c90 0x7f45640b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.722+0000 7f455be00640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45641478b0 0x7f45640b0c70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f454c00d5f0 tx=0x7f454c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.722+0000 7f4560c00640 1 -- 172.21.15.49:0/369078052 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0125f0 con 0x7f45641478b0 2024-04-15T15:20:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.722+0000 7f4560c00640 1 -- 172.21.15.49:0/369078052 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f454c019070 con 0x7f45641478b0 2024-04-15T15:20:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.722+0000 7f4569600640 1 -- 172.21.15.49:0/369078052 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4564143b80 con 0x7f45641478b0 2024-04-15T15:20:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.722+0000 7f4560c00640 1 -- 172.21.15.49:0/369078052 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0129d0 con 0x7f45641478b0 2024-04-15T15:20:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.722+0000 7f4569600640 1 -- 172.21.15.49:0/369078052 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4564144140 con 0x7f45641478b0 2024-04-15T15:20:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.723+0000 7f4569600640 1 -- 172.21.15.49:0/369078052 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456406c8c0 con 0x7f45641478b0 2024-04-15T15:20:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.725+0000 7f4560c00640 1 -- 172.21.15.49:0/369078052 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f454c012b70 con 0x7f45641478b0 2024-04-15T15:20:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.726+0000 7f4560c00640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f454006c8f0 0x7f454006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.726+0000 7f4560c00640 1 -- 172.21.15.49:0/369078052 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f454c08bf50 con 0x7f45641478b0 2024-04-15T15:20:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.726+0000 7f4563e00640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f454006c8f0 0x7f454006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.727+0000 7f4563e00640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f454006c8f0 0x7f454006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4554005880 tx=0x7f4554009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:12.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.731+0000 7f4560c00640 1 -- 172.21.15.49:0/369078052 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f456415e600 con 0x7f45641478b0 2024-04-15T15:20:12.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.969+0000 7f4569600640 1 -- 172.21.15.49:0/369078052 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_quiesce", "target": ["mon-mgr", ""]}) -- 0x7f4564148210 con 0x7f454006c8f0 2024-04-15T15:20:12.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.980+0000 7f4560c00640 1 -- 172.21.15.49:0/369078052 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+70 (secure 0 0 0) 0x7f4564148210 con 0x7f454006c8f0 2024-04-15T15:20:12.982 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:12.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.982+0000 7f4569600640 1 -- 172.21.15.49:0/369078052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f454006c8f0 msgr2=0x7f454006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.982+0000 7f4569600640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f454006c8f0 0x7f454006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4554005880 tx=0x7f4554009040 comp rx=0 tx=0).stop 2024-04-15T15:20:12.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.982+0000 7f4569600640 1 -- 172.21.15.49:0/369078052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45641478b0 msgr2=0x7f45640b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:12.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.982+0000 7f4569600640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45641478b0 0x7f45640b0c70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f454c00d5f0 tx=0x7f454c00dab0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.982+0000 7f4569600640 1 -- 172.21.15.49:0/369078052 shutdown_connections 2024-04-15T15:20:12.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.983+0000 7f4569600640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f454006c8f0 0x7f454006edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.983+0000 7f4569600640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4564147c90 0x7f45640b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.983+0000 7f4569600640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45641478b0 0x7f45640b0c70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.983+0000 7f4569600640 1 --2- 172.21.15.49:0/369078052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456406f750 0x7f456415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:12.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.983+0000 7f4569600640 1 -- 172.21.15.49:0/369078052 >> 172.21.15.49:0/369078052 conn(0x7f456406b930 msgr2=0x7f45641468e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:12.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.983+0000 7f4569600640 1 -- 172.21.15.49:0/369078052 shutdown_connections 2024-04-15T15:20:12.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:12.983+0000 7f4569600640 1 -- 172.21.15.49:0/369078052 wait complete. 2024-04-15T15:20:13.003 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:20:13.003 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:13.059 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:20:13.059 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:13.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:13.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.350+0000 7f65bcc00640 1 Processor -- start 2024-04-15T15:20:13.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.351+0000 7f65bcc00640 1 -- start start 2024-04-15T15:20:13.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.352+0000 7f65bcc00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f65b80d1c90 0x7f65b8162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.352+0000 7f65bcc00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b80d0090 con 0x7f65b80cf110 2024-04-15T15:20:13.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.352+0000 7f65bcc00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b80cfad0 con 0x7f65b8162020 2024-04-15T15:20:13.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.352+0000 7f65bcc00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f65b8163440 con 0x7f65b80d1c90 2024-04-15T15:20:13.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.353+0000 7f65aea00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f65b80d1c90 0x7f65b8162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.353+0000 7f65aea00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f65b80d1c90 0x7f65b8162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43062/0 (socket says 172.21.15.49:43062) 2024-04-15T15:20:13.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.353+0000 7f65aea00640 1 -- 172.21.15.49:0/3413665547 learned_addr learned my addr 172.21.15.49:0/3413665547 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:13.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.353+0000 7f65aea00640 1 -- 172.21.15.49:0/3413665547 >> v1:172.21.15.107:6789/0 conn(0x7f65b8162020 legacy=0x7f65b80d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.353+0000 7f65aea00640 1 -- 172.21.15.49:0/3413665547 >> v1:172.21.15.49:6789/0 conn(0x7f65b80cf110 legacy=0x7f65b80d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.353+0000 7f65aea00640 1 -- 172.21.15.49:0/3413665547 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b80cfdb0 con 0x7f65b80d1c90 2024-04-15T15:20:13.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.354+0000 7f65aea00640 1 --2- 172.21.15.49:0/3413665547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80d1c90 0x7f65b8162f00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6a199595c38836 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:13.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.355+0000 7f65b6a00640 1 -- 172.21.15.49:0/3413665547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a002f850 con 0x7f65b80d1c90 2024-04-15T15:20:13.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.355+0000 7f65b6a00640 1 -- 172.21.15.49:0/3413665547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a002f9f0 con 0x7f65b80d1c90 2024-04-15T15:20:13.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.355+0000 7f65b6a00640 1 -- 172.21.15.49:0/3413665547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a002fdc0 con 0x7f65b80d1c90 2024-04-15T15:20:13.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.356+0000 7f65bcc00640 1 -- 172.21.15.49:0/3413665547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80d1c90 msgr2=0x7f65b8162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.356+0000 7f65bcc00640 1 --2- 172.21.15.49:0/3413665547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80d1c90 0x7f65b8162f00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a002ba30 comp rx=0 tx=0).stop 2024-04-15T15:20:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.356+0000 7f65bcc00640 1 -- 172.21.15.49:0/3413665547 shutdown_connections 2024-04-15T15:20:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.356+0000 7f65bcc00640 1 --2- 172.21.15.49:0/3413665547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80d1c90 0x7f65b8162f00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.356+0000 7f65bcc00640 1 -- 172.21.15.49:0/3413665547 >> 172.21.15.49:0/3413665547 conn(0x7f65b806cf00 msgr2=0x7f65b806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.356+0000 7f65bcc00640 1 -- 172.21.15.49:0/3413665547 shutdown_connections 2024-04-15T15:20:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.356+0000 7f65bcc00640 1 -- 172.21.15.49:0/3413665547 wait complete. 2024-04-15T15:20:13.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.358+0000 7f65bcc00640 1 Processor -- start 2024-04-15T15:20:13.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.358+0000 7f65bcc00640 1 -- start start 2024-04-15T15:20:13.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.359+0000 7f65bcc00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80cf110 0x7f65b80cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.359+0000 7f65bcc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b80d1c90 0x7f65b814bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.359+0000 7f65bcc00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65b8162020 0x7f65b8154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.359+0000 7f65bcc00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f65b80cf880 con 0x7f65b80cf110 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.359+0000 7f65bcc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f65b80b7b30 con 0x7f65b80d1c90 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.359+0000 7f65bcc00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f65b80d2e70 con 0x7f65b8162020 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.359+0000 7f65b7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80cf110 0x7f65b80cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.360+0000 7f65b7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80cf110 0x7f65b80cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43076/0 (socket says 172.21.15.49:43076) 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.360+0000 7f65b7400640 1 -- 172.21.15.49:0/371384778 learned_addr learned my addr 172.21.15.49:0/371384778 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.360+0000 7f65aea00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b80d1c90 0x7f65b814bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.360+0000 7f65b7e00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65b8162020 0x7f65b8154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.360+0000 7f65b7400640 1 -- 172.21.15.49:0/371384778 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65b8162020 msgr2=0x7f65b8154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.360+0000 7f65b7400640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65b8162020 0x7f65b8154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.360+0000 7f65b7400640 1 -- 172.21.15.49:0/371384778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b80d1c90 msgr2=0x7f65b814bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.360+0000 7f65b7400640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b80d1c90 0x7f65b814bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.360+0000 7f65b7400640 1 -- 172.21.15.49:0/371384778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b80c9950 con 0x7f65b80cf110 2024-04-15T15:20:13.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.360+0000 7f65aea00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b80d1c90 0x7f65b814bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.361+0000 7f65b7400640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80cf110 0x7f65b80cd350 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f65a800c480 tx=0x7f65a800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.361+0000 7f65afe00640 1 -- 172.21.15.49:0/371384778 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a80052d0 con 0x7f65b80cf110 2024-04-15T15:20:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.361+0000 7f65afe00640 1 -- 172.21.15.49:0/371384778 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a8005470 con 0x7f65b80cf110 2024-04-15T15:20:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.361+0000 7f65afe00640 1 -- 172.21.15.49:0/371384778 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8012870 con 0x7f65b80cf110 2024-04-15T15:20:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.361+0000 7f65bcc00640 1 -- 172.21.15.49:0/371384778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b80c9c30 con 0x7f65b80cf110 2024-04-15T15:20:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.361+0000 7f65bcc00640 1 -- 172.21.15.49:0/371384778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b80ca1f0 con 0x7f65b80cf110 2024-04-15T15:20:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.362+0000 7f65bcc00640 1 -- 172.21.15.49:0/371384778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b80755c0 con 0x7f65b80cf110 2024-04-15T15:20:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.363+0000 7f65afe00640 1 -- 172.21.15.49:0/371384778 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65a8012a10 con 0x7f65b80cf110 2024-04-15T15:20:13.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.364+0000 7f65afe00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f659406c8f0 0x7f659406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.364+0000 7f65afe00640 1 -- 172.21.15.49:0/371384778 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f65a808b780 con 0x7f65b80cf110 2024-04-15T15:20:13.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.364+0000 7f65aea00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f659406c8f0 0x7f659406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.365+0000 7f65aea00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f659406c8f0 0x7f659406edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f65a0036000 tx=0x7f65a0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:13.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.369+0000 7f65afe00640 1 -- 172.21.15.49:0/371384778 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65a8059c20 con 0x7f65b80cf110 2024-04-15T15:20:13.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.439+0000 7f65afe00640 1 -- 172.21.15.49:0/371384778 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f65a8012d00 con 0x7f65b80cf110 2024-04-15T15:20:13.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.583+0000 7f65bcc00640 1 -- 172.21.15.49:0/371384778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65b80d25a0 con 0x7f65b80cf110 2024-04-15T15:20:13.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.586+0000 7f65afe00640 1 -- 172.21.15.49:0/371384778 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16669 (secure 0 0 0) 0x7f65a807cee0 con 0x7f65b80cf110 2024-04-15T15:20:13.588 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:13.589 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:01.591001+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:13.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 -- 172.21.15.49:0/371384778 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f659406c8f0 msgr2=0x7f659406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f659406c8f0 0x7f659406edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f65a0036000 tx=0x7f65a0008030 comp rx=0 tx=0).stop 2024-04-15T15:20:13.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 -- 172.21.15.49:0/371384778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80cf110 msgr2=0x7f65b80cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80cf110 0x7f65b80cd350 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f65a800c480 tx=0x7f65a800c940 comp rx=0 tx=0).stop 2024-04-15T15:20:13.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 -- 172.21.15.49:0/371384778 shutdown_connections 2024-04-15T15:20:13.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f659406c8f0 0x7f659406edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65b8162020 0x7f65b8154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65b80d1c90 0x7f65b814bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 --2- 172.21.15.49:0/371384778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65b80cf110 0x7f65b80cd350 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 -- 172.21.15.49:0/371384778 >> 172.21.15.49:0/371384778 conn(0x7f65b806cf00 msgr2=0x7f65b80d3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:13.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 -- 172.21.15.49:0/371384778 shutdown_connections 2024-04-15T15:20:13.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.589+0000 7f65bcc00640 1 -- 172.21.15.49:0/371384778 wait complete. 2024-04-15T15:20:13.613 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:58.063498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:59.009342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:13.614 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:13.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.843+0000 7f8f25a00640 1 Processor -- start 2024-04-15T15:20:13.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.843+0000 7f8f25a00640 1 -- start start 2024-04-15T15:20:13.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.845+0000 7f8f25a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 0x7f8f200b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.845+0000 7f8f25a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f200b5e00 con 0x7f8f200aef00 2024-04-15T15:20:13.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.845+0000 7f8f25a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f201483a0 con 0x7f8f20162b00 2024-04-15T15:20:13.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.845+0000 7f8f25a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f8f200afaa0 con 0x7f8f20147fc0 2024-04-15T15:20:13.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.846+0000 7f8f17e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8f200aef00 0x7f8f201623f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:40436/0 (socket says 172.21.15.49:40436) 2024-04-15T15:20:13.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.846+0000 7f8f17e00640 1 -- 172.21.15.49:0/347249618 learned_addr learned my addr 172.21.15.49:0/347249618 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:13.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.846+0000 7f8f1fe00640 1 --2- 172.21.15.49:0/347249618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 0x7f8f200b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.846+0000 7f8f1f400640 1 -- 172.21.15.49:0/347249618 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4278350880 0 0) 0x7f8f200b5e00 con 0x7f8f200aef00 2024-04-15T15:20:13.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.847+0000 7f8f1f400640 1 -- 172.21.15.49:0/347249618 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f04003520 con 0x7f8f200aef00 2024-04-15T15:20:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.847+0000 7f8f1f400640 1 -- 172.21.15.49:0/347249618 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1387783982 0 0) 0x7f8f201483a0 con 0x7f8f20162b00 2024-04-15T15:20:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.847+0000 7f8f1f400640 1 -- 172.21.15.49:0/347249618 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f200b5e00 con 0x7f8f20162b00 2024-04-15T15:20:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.847+0000 7f8f1fe00640 1 -- 172.21.15.49:0/347249618 >> v1:172.21.15.107:6789/0 conn(0x7f8f20162b00 legacy=0x7f8f2006df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.847+0000 7f8f1fe00640 1 -- 172.21.15.49:0/347249618 >> v1:172.21.15.49:6789/0 conn(0x7f8f200aef00 legacy=0x7f8f201623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.847+0000 7f8f1f400640 1 -- 172.21.15.49:0/347249618 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 351693603 0 0) 0x7f8f04003520 con 0x7f8f200aef00 2024-04-15T15:20:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.847+0000 7f8f1fe00640 1 -- 172.21.15.49:0/347249618 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f200afd80 con 0x7f8f20147fc0 2024-04-15T15:20:13.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.849+0000 7f8f1fe00640 1 --2- 172.21.15.49:0/347249618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 0x7f8f200b36a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f100097d0 tx=0x7f8f1002b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ecf1047dd30f4a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:13.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.849+0000 7f8f1f400640 1 -- 172.21.15.49:0/347249618 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10037070 con 0x7f8f20147fc0 2024-04-15T15:20:13.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.849+0000 7f8f1f400640 1 -- 172.21.15.49:0/347249618 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f10030510 con 0x7f8f20147fc0 2024-04-15T15:20:13.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.849+0000 7f8f1f400640 1 -- 172.21.15.49:0/347249618 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08001670 con 0x7f8f20147fc0 2024-04-15T15:20:13.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.850+0000 7f8f25a00640 1 -- 172.21.15.49:0/347249618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 msgr2=0x7f8f200b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.850+0000 7f8f25a00640 1 --2- 172.21.15.49:0/347249618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 0x7f8f200b36a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f100097d0 tx=0x7f8f1002b770 comp rx=0 tx=0).stop 2024-04-15T15:20:13.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.850+0000 7f8f25a00640 1 -- 172.21.15.49:0/347249618 shutdown_connections 2024-04-15T15:20:13.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.850+0000 7f8f25a00640 1 --2- 172.21.15.49:0/347249618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 0x7f8f200b36a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.850+0000 7f8f25a00640 1 -- 172.21.15.49:0/347249618 >> 172.21.15.49:0/347249618 conn(0x7f8f200b23a0 msgr2=0x7f8f200b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:13.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.850+0000 7f8f25a00640 1 -- 172.21.15.49:0/347249618 shutdown_connections 2024-04-15T15:20:13.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.850+0000 7f8f25a00640 1 -- 172.21.15.49:0/347249618 wait complete. 2024-04-15T15:20:13.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.852+0000 7f8f25a00640 1 Processor -- start 2024-04-15T15:20:13.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.852+0000 7f8f25a00640 1 -- start start 2024-04-15T15:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.853+0000 7f8f25a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f200aef00 0x7f8f2015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.853+0000 7f8f25a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 0x7f8f20143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.853+0000 7f8f25a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f20162b00 0x7f8f201428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.853+0000 7f8f25a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f200b3be0 con 0x7f8f200aef00 2024-04-15T15:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.853+0000 7f8f25a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f200b0060 con 0x7f8f20162b00 2024-04-15T15:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.853+0000 7f8f25a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f8f200b3d60 con 0x7f8f20147fc0 2024-04-15T15:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.853+0000 7f8f1fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f200aef00 0x7f8f2015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.854+0000 7f8f1fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f200aef00 0x7f8f2015e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43078/0 (socket says 172.21.15.49:43078) 2024-04-15T15:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.854+0000 7f8f1fe00640 1 -- 172.21.15.49:0/1482932615 learned_addr learned my addr 172.21.15.49:0/1482932615 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.854+0000 7f8f17e00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 0x7f8f20143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.854+0000 7f8f25000640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f20162b00 0x7f8f201428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.854+0000 7f8f1fe00640 1 -- 172.21.15.49:0/1482932615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 msgr2=0x7f8f20143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.854+0000 7f8f1fe00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 0x7f8f20143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.854+0000 7f8f1fe00640 1 -- 172.21.15.49:0/1482932615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f20162b00 msgr2=0x7f8f201428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.854+0000 7f8f1fe00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f20162b00 0x7f8f201428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.854+0000 7f8f1fe00640 1 -- 172.21.15.49:0/1482932615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f201f7ca0 con 0x7f8f200aef00 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.854+0000 7f8f1fe00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f200aef00 0x7f8f2015e1f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8f10000c00 tx=0x7f8f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.855+0000 7f8f1cc00640 1 -- 172.21.15.49:0/1482932615 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10041020 con 0x7f8f200aef00 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.855+0000 7f8f1cc00640 1 -- 172.21.15.49:0/1482932615 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1003c870 con 0x7f8f200aef00 2024-04-15T15:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.855+0000 7f8f1cc00640 1 -- 172.21.15.49:0/1482932615 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10037040 con 0x7f8f200aef00 2024-04-15T15:20:13.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.855+0000 7f8f25a00640 1 -- 172.21.15.49:0/1482932615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f201f7f80 con 0x7f8f200aef00 2024-04-15T15:20:13.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.855+0000 7f8f25a00640 1 -- 172.21.15.49:0/1482932615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f201f8540 con 0x7f8f200aef00 2024-04-15T15:20:13.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.856+0000 7f8f25a00640 1 -- 172.21.15.49:0/1482932615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f200755c0 con 0x7f8f200aef00 2024-04-15T15:20:13.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.857+0000 7f8f1cc00640 1 -- 172.21.15.49:0/1482932615 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8f10044020 con 0x7f8f200aef00 2024-04-15T15:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.859+0000 7f8f1cc00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8efc06c980 0x7f8efc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.859+0000 7f8f1cc00640 1 -- 172.21.15.49:0/1482932615 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8f100ad6e0 con 0x7f8f200aef00 2024-04-15T15:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.859+0000 7f8f17e00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8efc06c980 0x7f8efc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:13.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.860+0000 7f8f17e00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8efc06c980 0x7f8efc06ee40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8f201434d0 tx=0x7f8f08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:13.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:13.864+0000 7f8f1cc00640 1 -- 172.21.15.49:0/1482932615 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f1004b030 con 0x7f8f200aef00 2024-04-15T15:20:14.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.066+0000 7f8f25a00640 1 -- 172.21.15.49:0/1482932615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8f2006ff20 con 0x7f8f200aef00 2024-04-15T15:20:14.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.067+0000 7f8f1cc00640 1 -- 172.21.15.49:0/1482932615 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f8f1007ba10 con 0x7f8f200aef00 2024-04-15T15:20:14.069 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:20:14.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.069+0000 7f8f25a00640 1 -- 172.21.15.49:0/1482932615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8efc06c980 msgr2=0x7f8efc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.069+0000 7f8f25a00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8efc06c980 0x7f8efc06ee40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8f201434d0 tx=0x7f8f08009040 comp rx=0 tx=0).stop 2024-04-15T15:20:14.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.069+0000 7f8f25a00640 1 -- 172.21.15.49:0/1482932615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f200aef00 msgr2=0x7f8f2015e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.069+0000 7f8f25a00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f200aef00 0x7f8f2015e1f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8f10000c00 tx=0x7f8f100046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.070+0000 7f8f25a00640 1 -- 172.21.15.49:0/1482932615 shutdown_connections 2024-04-15T15:20:14.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.070+0000 7f8f25a00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8efc06c980 0x7f8efc06ee40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.070+0000 7f8f25a00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f20147fc0 0x7f8f20143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.070+0000 7f8f25a00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f20162b00 0x7f8f201428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.070+0000 7f8f25a00640 1 --2- 172.21.15.49:0/1482932615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f200aef00 0x7f8f2015e1f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.070+0000 7f8f25a00640 1 -- 172.21.15.49:0/1482932615 >> 172.21.15.49:0/1482932615 conn(0x7f8f200b23a0 msgr2=0x7f8f200b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:14.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.070+0000 7f8f25a00640 1 -- 172.21.15.49:0/1482932615 shutdown_connections 2024-04-15T15:20:14.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.070+0000 7f8f25a00640 1 -- 172.21.15.49:0/1482932615 wait complete. 2024-04-15T15:20:14.092 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:14.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.320+0000 7f3f73000640 1 Processor -- start 2024-04-15T15:20:14.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.321+0000 7f3f73000640 1 -- start start 2024-04-15T15:20:14.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.322+0000 7f3f73000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3f6c0725f0 0x7f3f6c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.322+0000 7f3f73000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3f6c0ce9e0 0x7f3f6c06df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.322+0000 7f3f73000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f6c0cd2d0 con 0x7f3f6c1461c0 2024-04-15T15:20:14.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.322+0000 7f3f73000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3f6c0cf1d0 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.322+0000 7f3f73000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f3f6c0cd080 con 0x7f3f6c0725f0 2024-04-15T15:20:14.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.323+0000 7f3f72600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3f6c0ce9e0 0x7f3f6c06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.323+0000 7f3f71200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3f6c0725f0 0x7f3f6c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.323+0000 7f3f72600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3f6c0ce9e0 0x7f3f6c06df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33034/0 (socket says 172.21.15.49:33034) 2024-04-15T15:20:14.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.323+0000 7f3f72600640 1 -- 172.21.15.49:0/3085757879 learned_addr learned my addr 172.21.15.49:0/3085757879 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:14.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.324+0000 7f3f72600640 1 -- 172.21.15.49:0/3085757879 >> v2:172.21.15.107:3300/0 conn(0x7f3f6c0725f0 msgr2=0x7f3f6c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:14.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.324+0000 7f3f72600640 1 --2- 172.21.15.49:0/3085757879 >> v2:172.21.15.107:3300/0 conn(0x7f3f6c0725f0 0x7f3f6c0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.324+0000 7f3f72600640 1 -- 172.21.15.49:0/3085757879 >> v1:172.21.15.107:6789/0 conn(0x7f3f6c1461c0 legacy=0x7f3f6c071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.324+0000 7f3f72600640 1 -- 172.21.15.49:0/3085757879 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6c0cd5b0 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.325+0000 7f3f72600640 1 --2- 172.21.15.49:0/3085757879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 0x7f3f6c06df20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0737f0 tx=0x7f3f6402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0c6c40570072f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.325+0000 7f3f6be00640 1 -- 172.21.15.49:0/3085757879 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6402cb10 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.325+0000 7f3f6be00640 1 -- 172.21.15.49:0/3085757879 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6402ccb0 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.325+0000 7f3f6be00640 1 -- 172.21.15.49:0/3085757879 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6402ce60 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.326+0000 7f3f73000640 1 -- 172.21.15.49:0/3085757879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 msgr2=0x7f3f6c06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.326+0000 7f3f73000640 1 --2- 172.21.15.49:0/3085757879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 0x7f3f6c06df20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0737f0 tx=0x7f3f6402bf70 comp rx=0 tx=0).stop 2024-04-15T15:20:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.326+0000 7f3f73000640 1 -- 172.21.15.49:0/3085757879 shutdown_connections 2024-04-15T15:20:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.326+0000 7f3f73000640 1 --2- 172.21.15.49:0/3085757879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 0x7f3f6c06df20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.326+0000 7f3f73000640 1 --2- 172.21.15.49:0/3085757879 >> v2:172.21.15.107:3300/0 conn(0x7f3f6c0725f0 0x7f3f6c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.326+0000 7f3f73000640 1 -- 172.21.15.49:0/3085757879 >> 172.21.15.49:0/3085757879 conn(0x7f3f6c06b930 msgr2=0x7f3f6c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.326+0000 7f3f73000640 1 -- 172.21.15.49:0/3085757879 shutdown_connections 2024-04-15T15:20:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.326+0000 7f3f73000640 1 -- 172.21.15.49:0/3085757879 wait complete. 2024-04-15T15:20:14.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.328+0000 7f3f73000640 1 Processor -- start 2024-04-15T15:20:14.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.328+0000 7f3f73000640 1 -- start start 2024-04-15T15:20:14.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.329+0000 7f3f73000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f6c0725f0 0x7f3f6c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.329+0000 7f3f73000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 0x7f3f6c144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.329+0000 7f3f73000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f6c1461c0 0x7f3f6c15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.329+0000 7f3f73000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c0cee30 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.329+0000 7f3f73000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c0cdb70 con 0x7f3f6c1461c0 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.329+0000 7f3f73000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f3f6c0cefb0 con 0x7f3f6c0725f0 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.330+0000 7f3f71200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 0x7f3f6c144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.330+0000 7f3f71200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 0x7f3f6c144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33048/0 (socket says 172.21.15.49:33048) 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.330+0000 7f3f71200640 1 -- 172.21.15.49:0/4008163797 learned_addr learned my addr 172.21.15.49:0/4008163797 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.330+0000 7f3f72600640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f6c1461c0 0x7f3f6c15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.330+0000 7f3f71c00640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f6c0725f0 0x7f3f6c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.330+0000 7f3f71200640 1 -- 172.21.15.49:0/4008163797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f6c0725f0 msgr2=0x7f3f6c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.330+0000 7f3f71200640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f6c0725f0 0x7f3f6c156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.330+0000 7f3f71200640 1 -- 172.21.15.49:0/4008163797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f6c1461c0 msgr2=0x7f3f6c15e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.330+0000 7f3f71200640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f6c1461c0 0x7f3f6c15e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.330+0000 7f3f71200640 1 -- 172.21.15.49:0/4008163797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6c143d70 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.331+0000 7f3f71200640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 0x7f3f6c144e10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800e8a0 tx=0x7f3f5800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.331+0000 7f3f69600640 1 -- 172.21.15.49:0/4008163797 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58017070 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.331+0000 7f3f69600640 1 -- 172.21.15.49:0/4008163797 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f5800c940 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.331+0000 7f3f69600640 1 -- 172.21.15.49:0/4008163797 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800cd30 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.331+0000 7f3f73000640 1 -- 172.21.15.49:0/4008163797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6c144050 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.331+0000 7f3f73000640 1 -- 172.21.15.49:0/4008163797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6c144610 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.332+0000 7f3f73000640 1 -- 172.21.15.49:0/4008163797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f6c0755c0 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.333+0000 7f3f69600640 1 -- 172.21.15.49:0/4008163797 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3f5801a070 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.334+0000 7f3f69600640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3f4c06c980 0x7f3f4c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.334+0000 7f3f69600640 1 -- 172.21.15.49:0/4008163797 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3f5808be40 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.335+0000 7f3f71c00640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3f4c06c980 0x7f3f4c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.335+0000 7f3f71c00640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3f4c06c980 0x7f3f4c06ee40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f60001640 tx=0x7f3f60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.339+0000 7f3f69600640 1 -- 172.21.15.49:0/4008163797 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f5805a1f0 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.546+0000 7f3f73000640 1 -- 172.21.15.49:0/4008163797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3f6c156550 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.547+0000 7f3f69600640 1 -- 172.21.15.49:0/4008163797 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f3f58059af0 con 0x7f3f6c0ce9e0 2024-04-15T15:20:14.549 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.549+0000 7f3f73000640 1 -- 172.21.15.49:0/4008163797 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3f4c06c980 msgr2=0x7f3f4c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3f4c06c980 0x7f3f4c06ee40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f60001640 tx=0x7f3f60009040 comp rx=0 tx=0).stop 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 -- 172.21.15.49:0/4008163797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 msgr2=0x7f3f6c144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 0x7f3f6c144e10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800e8a0 tx=0x7f3f5800ed60 comp rx=0 tx=0).stop 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 -- 172.21.15.49:0/4008163797 shutdown_connections 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3f4c06c980 0x7f3f4c06ee40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f6c1461c0 0x7f3f6c15e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f6c0ce9e0 0x7f3f6c144e10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 --2- 172.21.15.49:0/4008163797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f6c0725f0 0x7f3f6c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 -- 172.21.15.49:0/4008163797 >> 172.21.15.49:0/4008163797 conn(0x7f3f6c06b930 msgr2=0x7f3f6c15c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 -- 172.21.15.49:0/4008163797 shutdown_connections 2024-04-15T15:20:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.550+0000 7f3f73000640 1 -- 172.21.15.49:0/4008163797 wait complete. 2024-04-15T15:20:14.571 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.807+0000 7f9393200640 1 Processor -- start 2024-04-15T15:20:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.807+0000 7f9393200640 1 -- start start 2024-04-15T15:20:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.808+0000 7f9393200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93941461c0 0x7f9394071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.808+0000 7f9393200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9394072420 0x7f93940ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.809+0000 7f9393200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93940cd2d0 con 0x7f93940ce9e0 2024-04-15T15:20:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.809+0000 7f9393200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f93940cf1d0 con 0x7f9394072420 2024-04-15T15:20:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.809+0000 7f9393200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9394072880 con 0x7f93941461c0 2024-04-15T15:20:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.809+0000 7f9391e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93941461c0 0x7f9394071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.809+0000 7f9391400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9394072420 0x7f93940ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.809+0000 7f9391e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93941461c0 0x7f9394071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:44310/0 (socket says 172.21.15.49:44310) 2024-04-15T15:20:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.809+0000 7f9391e00640 1 -- 172.21.15.49:0/324700214 learned_addr learned my addr 172.21.15.49:0/324700214 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.810+0000 7f9391400640 1 -- 172.21.15.49:0/324700214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93941461c0 msgr2=0x7f9394071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.810+0000 7f9391400640 1 --2- 172.21.15.49:0/324700214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93941461c0 0x7f9394071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.810+0000 7f9391400640 1 -- 172.21.15.49:0/324700214 >> v1:172.21.15.107:6789/0 conn(0x7f93940ce9e0 legacy=0x7f9394160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.810+0000 7f9391400640 1 -- 172.21.15.49:0/324700214 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93940cd5b0 con 0x7f9394072420 2024-04-15T15:20:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.812+0000 7f9391400640 1 --2- 172.21.15.49:0/324700214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394072420 0x7f93940ce4a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9384009d90 tx=0x7f938402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd1dc6fbe7724e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:14.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.813+0000 7f9390a00640 1 -- 172.21.15.49:0/324700214 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384039070 con 0x7f9394072420 2024-04-15T15:20:14.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.813+0000 7f9390a00640 1 -- 172.21.15.49:0/324700214 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938402cc50 con 0x7f9394072420 2024-04-15T15:20:14.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.813+0000 7f9390a00640 1 -- 172.21.15.49:0/324700214 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384034440 con 0x7f9394072420 2024-04-15T15:20:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.813+0000 7f9393200640 1 -- 172.21.15.49:0/324700214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394072420 msgr2=0x7f93940ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.813+0000 7f9393200640 1 --2- 172.21.15.49:0/324700214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394072420 0x7f93940ce4a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9384009d90 tx=0x7f938402bea0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.814+0000 7f9393200640 1 -- 172.21.15.49:0/324700214 shutdown_connections 2024-04-15T15:20:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.814+0000 7f9393200640 1 --2- 172.21.15.49:0/324700214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394072420 0x7f93940ce4a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.814+0000 7f9393200640 1 --2- 172.21.15.49:0/324700214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93941461c0 0x7f9394071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.814+0000 7f9393200640 1 -- 172.21.15.49:0/324700214 >> 172.21.15.49:0/324700214 conn(0x7f939406b930 msgr2=0x7f939406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.814+0000 7f9393200640 1 -- 172.21.15.49:0/324700214 shutdown_connections 2024-04-15T15:20:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.814+0000 7f9393200640 1 -- 172.21.15.49:0/324700214 wait complete. 2024-04-15T15:20:14.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.816+0000 7f9393200640 1 Processor -- start 2024-04-15T15:20:14.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.816+0000 7f9393200640 1 -- start start 2024-04-15T15:20:14.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.817+0000 7f9393200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9394072420 0x7f93941f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.817+0000 7f9393200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93940ce9e0 0x7f939415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.817+0000 7f9393200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93941461c0 0x7f939415bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.817+0000 7f9393200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f93940cdb70 con 0x7f93941461c0 2024-04-15T15:20:14.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.817+0000 7f9393200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93940cd080 con 0x7f93940ce9e0 2024-04-15T15:20:14.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.817+0000 7f9393200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f93940cee70 con 0x7f9394072420 2024-04-15T15:20:14.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.818+0000 7f9392800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93941461c0 0x7f939415bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.818+0000 7f9392800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93941461c0 0x7f939415bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33050/0 (socket says 172.21.15.49:33050) 2024-04-15T15:20:14.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.818+0000 7f9392800640 1 -- 172.21.15.49:0/3309656337 learned_addr learned my addr 172.21.15.49:0/3309656337 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:14.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.818+0000 7f9391e00640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9394072420 0x7f93941f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.818+0000 7f9391400640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93940ce9e0 0x7f939415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.818+0000 7f9392800640 1 -- 172.21.15.49:0/3309656337 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9394072420 msgr2=0x7f93941f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.818+0000 7f9392800640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9394072420 0x7f93941f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.818+0000 7f9392800640 1 -- 172.21.15.49:0/3309656337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93940ce9e0 msgr2=0x7f939415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.818+0000 7f9392800640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93940ce9e0 0x7f939415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.818+0000 7f9392800640 1 -- 172.21.15.49:0/3309656337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93940b4740 con 0x7f93941461c0 2024-04-15T15:20:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.819+0000 7f9392800640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93941461c0 0x7f939415bc20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f938c001eb0 tx=0x7f938c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.819+0000 7f938a000640 1 -- 172.21.15.49:0/3309656337 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c017070 con 0x7f93941461c0 2024-04-15T15:20:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.819+0000 7f938a000640 1 -- 172.21.15.49:0/3309656337 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938c0052d0 con 0x7f93941461c0 2024-04-15T15:20:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.819+0000 7f9393200640 1 -- 172.21.15.49:0/3309656337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93940b4a20 con 0x7f93941461c0 2024-04-15T15:20:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.819+0000 7f938a000640 1 -- 172.21.15.49:0/3309656337 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c003810 con 0x7f93941461c0 2024-04-15T15:20:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.819+0000 7f9393200640 1 -- 172.21.15.49:0/3309656337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93940b4fe0 con 0x7f93941461c0 2024-04-15T15:20:14.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.821+0000 7f9393200640 1 -- 172.21.15.49:0/3309656337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93940755c0 con 0x7f93941461c0 2024-04-15T15:20:14.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.821+0000 7f938a000640 1 -- 172.21.15.49:0/3309656337 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f938c01a070 con 0x7f93941461c0 2024-04-15T15:20:14.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.822+0000 7f938a000640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f937006c980 0x7f937006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:14.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.823+0000 7f938a000640 1 -- 172.21.15.49:0/3309656337 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f938c08af30 con 0x7f93941461c0 2024-04-15T15:20:14.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.823+0000 7f9391e00640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f937006c980 0x7f937006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:14.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.824+0000 7f9391e00640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f937006c980 0x7f937006ee40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f937c009ac0 tx=0x7f937c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:14.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:14.828+0000 7f938a000640 1 -- 172.21.15.49:0/3309656337 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f938c059370 con 0x7f93941461c0 2024-04-15T15:20:15.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.035+0000 7f9393200640 1 -- 172.21.15.49:0/3309656337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9394072e40 con 0x7f93941461c0 2024-04-15T15:20:15.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.036+0000 7f938a000640 1 -- 172.21.15.49:0/3309656337 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f938c05f020 con 0x7f93941461c0 2024-04-15T15:20:15.041 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:20:15.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.039+0000 7f9393200640 1 -- 172.21.15.49:0/3309656337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f937006c980 msgr2=0x7f937006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:15.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.039+0000 7f9393200640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f937006c980 0x7f937006ee40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f937c009ac0 tx=0x7f937c008040 comp rx=0 tx=0).stop 2024-04-15T15:20:15.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.039+0000 7f9393200640 1 -- 172.21.15.49:0/3309656337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93941461c0 msgr2=0x7f939415bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:15.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.039+0000 7f9393200640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93941461c0 0x7f939415bc20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f938c001eb0 tx=0x7f938c00c890 comp rx=0 tx=0).stop 2024-04-15T15:20:15.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.039+0000 7f9393200640 1 -- 172.21.15.49:0/3309656337 shutdown_connections 2024-04-15T15:20:15.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.039+0000 7f9393200640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f937006c980 0x7f937006ee40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.039+0000 7f9393200640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93941461c0 0x7f939415bc20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.039+0000 7f9393200640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93940ce9e0 0x7f939415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.039+0000 7f9393200640 1 --2- 172.21.15.49:0/3309656337 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9394072420 0x7f93941f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.039+0000 7f9393200640 1 -- 172.21.15.49:0/3309656337 >> 172.21.15.49:0/3309656337 conn(0x7f939406b930 msgr2=0x7f93940b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:15.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.040+0000 7f9393200640 1 -- 172.21.15.49:0/3309656337 shutdown_connections 2024-04-15T15:20:15.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.040+0000 7f9393200640 1 -- 172.21.15.49:0/3309656337 wait complete. 2024-04-15T15:20:15.063 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.278+0000 7f28d2600640 1 Processor -- start 2024-04-15T15:20:15.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.279+0000 7f28d2600640 1 -- start start 2024-04-15T15:20:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.280+0000 7f28d2600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f28cc162dd0 0x7f28cc0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.280+0000 7f28d2600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b5530 0x7f28cc0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.280+0000 7f28d2600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28cc0ce760 con 0x7f28cc0b5b00 2024-04-15T15:20:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.280+0000 7f28d2600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f28cc0cea40 con 0x7f28cc162dd0 2024-04-15T15:20:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.280+0000 7f28d2600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc0ced20 con 0x7f28cc0b5530 2024-04-15T15:20:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.281+0000 7f28cbe00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f28cc162dd0 0x7f28cc0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.281+0000 7f28d1c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b5530 0x7f28cc0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.281+0000 7f28d1c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b5530 0x7f28cc0b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:44328/0 (socket says 172.21.15.49:44328) 2024-04-15T15:20:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.281+0000 7f28d1c00640 1 -- 172.21.15.49:0/3070989178 learned_addr learned my addr 172.21.15.49:0/3070989178 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.282+0000 7f28cbe00640 1 -- 172.21.15.49:0/3070989178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b5530 msgr2=0x7f28cc0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.282+0000 7f28cbe00640 1 --2- 172.21.15.49:0/3070989178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b5530 0x7f28cc0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.282+0000 7f28cbe00640 1 -- 172.21.15.49:0/3070989178 >> v1:172.21.15.107:6789/0 conn(0x7f28cc0b5b00 legacy=0x7f28cc0b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.282+0000 7f28cbe00640 1 -- 172.21.15.49:0/3070989178 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc0b4b90 con 0x7f28cc162dd0 2024-04-15T15:20:15.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.282+0000 7f28d1c00640 1 --2- 172.21.15.49:0/3070989178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b5530 0x7f28cc0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:15.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.283+0000 7f28cbe00640 1 --2- 172.21.15.49:0/3070989178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28cc162dd0 0x7f28cc0afdf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f28b80097a0 tx=0x7f28b802b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc355304f0e900d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:15.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.284+0000 7f28cb400640 1 -- 172.21.15.49:0/3070989178 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8037070 con 0x7f28cc162dd0 2024-04-15T15:20:15.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.284+0000 7f28cb400640 1 -- 172.21.15.49:0/3070989178 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b8030710 con 0x7f28cc162dd0 2024-04-15T15:20:15.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.284+0000 7f28cb400640 1 -- 172.21.15.49:0/3070989178 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8030ae0 con 0x7f28cc162dd0 2024-04-15T15:20:15.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.284+0000 7f28d2600640 1 -- 172.21.15.49:0/3070989178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28cc162dd0 msgr2=0x7f28cc0afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:15.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.284+0000 7f28d2600640 1 --2- 172.21.15.49:0/3070989178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28cc162dd0 0x7f28cc0afdf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f28b80097a0 tx=0x7f28b802b710 comp rx=0 tx=0).stop 2024-04-15T15:20:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.285+0000 7f28d2600640 1 -- 172.21.15.49:0/3070989178 shutdown_connections 2024-04-15T15:20:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.285+0000 7f28d2600640 1 --2- 172.21.15.49:0/3070989178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b5530 0x7f28cc0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.285+0000 7f28d2600640 1 --2- 172.21.15.49:0/3070989178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28cc162dd0 0x7f28cc0afdf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.285+0000 7f28d2600640 1 -- 172.21.15.49:0/3070989178 >> 172.21.15.49:0/3070989178 conn(0x7f28cc0b23a0 msgr2=0x7f28cc0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.285+0000 7f28d2600640 1 -- 172.21.15.49:0/3070989178 shutdown_connections 2024-04-15T15:20:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.285+0000 7f28d2600640 1 -- 172.21.15.49:0/3070989178 wait complete. 2024-04-15T15:20:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.286+0000 7f28d2600640 1 Processor -- start 2024-04-15T15:20:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.287+0000 7f28d2600640 1 -- start start 2024-04-15T15:20:15.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28d2600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0b5530 0x7f28cc1eda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:15.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28d2600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28cc0b5b00 0x7f28cc158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:15.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28d2600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc162dd0 0x7f28cc15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:15.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28d2600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0cf6a0 con 0x7f28cc0b5530 2024-04-15T15:20:15.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28d2600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0cf520 con 0x7f28cc0b5b00 2024-04-15T15:20:15.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28d2600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc0cf820 con 0x7f28cc162dd0 2024-04-15T15:20:15.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28d1200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0b5530 0x7f28cc1eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:15.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28d1200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0b5530 0x7f28cc1eda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33056/0 (socket says 172.21.15.49:33056) 2024-04-15T15:20:15.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28d1200640 1 -- 172.21.15.49:0/2170919122 learned_addr learned my addr 172.21.15.49:0/2170919122 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:15.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28d1c00640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc162dd0 0x7f28cc15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:15.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.288+0000 7f28cbe00640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28cc0b5b00 0x7f28cc158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:15.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28d1200640 1 -- 172.21.15.49:0/2170919122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc162dd0 msgr2=0x7f28cc15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:15.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28d1200640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc162dd0 0x7f28cc15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28d1200640 1 -- 172.21.15.49:0/2170919122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28cc0b5b00 msgr2=0x7f28cc158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:15.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28d1200640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28cc0b5b00 0x7f28cc158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28d1200640 1 -- 172.21.15.49:0/2170919122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc158f90 con 0x7f28cc0b5530 2024-04-15T15:20:15.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28d1200640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0b5530 0x7f28cc1eda40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c780 tx=0x7f28c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:15.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28c8c00640 1 -- 172.21.15.49:0/2170919122 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0017070 con 0x7f28cc0b5530 2024-04-15T15:20:15.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28c8c00640 1 -- 172.21.15.49:0/2170919122 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c0005290 con 0x7f28cc0b5530 2024-04-15T15:20:15.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28c8c00640 1 -- 172.21.15.49:0/2170919122 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c000a880 con 0x7f28cc0b5530 2024-04-15T15:20:15.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28d2600640 1 -- 172.21.15.49:0/2170919122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc1ec9a0 con 0x7f28cc0b5530 2024-04-15T15:20:15.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.289+0000 7f28d2600640 1 -- 172.21.15.49:0/2170919122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc1ecf60 con 0x7f28cc0b5530 2024-04-15T15:20:15.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.291+0000 7f28d2600640 1 -- 172.21.15.49:0/2170919122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc0755c0 con 0x7f28cc0b5530 2024-04-15T15:20:15.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.292+0000 7f28c8c00640 1 -- 172.21.15.49:0/2170919122 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f28c001a070 con 0x7f28cc0b5530 2024-04-15T15:20:15.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.293+0000 7f28c8c00640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f28ac06c980 0x7f28ac06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:15.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.293+0000 7f28c8c00640 1 -- 172.21.15.49:0/2170919122 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f28c008b140 con 0x7f28cc0b5530 2024-04-15T15:20:15.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.293+0000 7f28cbe00640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f28ac06c980 0x7f28ac06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:15.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.294+0000 7f28cbe00640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f28ac06c980 0x7f28ac06ee40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f28b8034000 tx=0x7f28b8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:15.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.297+0000 7f28c8c00640 1 -- 172.21.15.49:0/2170919122 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28cc158f90 con 0x7f28cc0b5530 2024-04-15T15:20:15.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.520+0000 7f28d2600640 1 -- 172.21.15.49:0/2170919122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f28cc0cd720 con 0x7f28cc0b5530 2024-04-15T15:20:15.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.521+0000 7f28c8c00640 1 -- 172.21.15.49:0/2170919122 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f28c0059510 con 0x7f28cc0b5530 2024-04-15T15:20:15.522 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2024-04-15T15:20:15.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 -- 172.21.15.49:0/2170919122 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f28ac06c980 msgr2=0x7f28ac06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:15.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f28ac06c980 0x7f28ac06ee40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f28b8034000 tx=0x7f28b8008030 comp rx=0 tx=0).stop 2024-04-15T15:20:15.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 -- 172.21.15.49:0/2170919122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0b5530 msgr2=0x7f28cc1eda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:15.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0b5530 0x7f28cc1eda40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c780 tx=0x7f28c000cc40 comp rx=0 tx=0).stop 2024-04-15T15:20:15.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 -- 172.21.15.49:0/2170919122 shutdown_connections 2024-04-15T15:20:15.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f28ac06c980 0x7f28ac06ee40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc162dd0 0x7f28cc15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28cc0b5b00 0x7f28cc158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 --2- 172.21.15.49:0/2170919122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0b5530 0x7f28cc1eda40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:15.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 -- 172.21.15.49:0/2170919122 >> 172.21.15.49:0/2170919122 conn(0x7f28cc0b23a0 msgr2=0x7f28cc0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:15.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 -- 172.21.15.49:0/2170919122 shutdown_connections 2024-04-15T15:20:15.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:15.524+0000 7f28d2600640 1 -- 172.21.15.49:0/2170919122 wait complete. 2024-04-15T15:20:15.545 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-15T15:20:15.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:15.545 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:15.664 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:20:15.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:15.665 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-15T15:20:15.716 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:20:15.716 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:15.777 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:20:15.777 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:15.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.073+0000 7fa4b6c00640 1 Processor -- start 2024-04-15T15:20:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.073+0000 7fa4b6c00640 1 -- start start 2024-04-15T15:20:16.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.075+0000 7fa4b6c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b015b860 0x7fa4b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.075+0000 7fa4b6c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa4b006f320 0x7fa4b00b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.075+0000 7fa4b6c00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b00cdfb0 con 0x7fa4b015b480 2024-04-15T15:20:16.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.075+0000 7fa4b6c00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b015f060 con 0x7fa4b006f320 2024-04-15T15:20:16.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.075+0000 7fa4b6c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b015fec0 con 0x7fa4b015b860 2024-04-15T15:20:16.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.075+0000 7fa4b6200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa4b006f320 0x7fa4b00b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.075+0000 7fa4b6200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa4b006f320 0x7fa4b00b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33072/0 (socket says 172.21.15.49:33072) 2024-04-15T15:20:16.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.075+0000 7fa4b6200640 1 -- 172.21.15.49:0/1672894613 learned_addr learned my addr 172.21.15.49:0/1672894613 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:16.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.075+0000 7fa4b4e00640 1 --2- 172.21.15.49:0/1672894613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b015b860 0x7fa4b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.076+0000 7fa4b6200640 1 -- 172.21.15.49:0/1672894613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b015b860 msgr2=0x7fa4b006ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:16.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.076+0000 7fa4b6200640 1 --2- 172.21.15.49:0/1672894613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b015b860 0x7fa4b006ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.076+0000 7fa4b6200640 1 -- 172.21.15.49:0/1672894613 >> v1:172.21.15.107:6789/0 conn(0x7fa4b015b480 legacy=0x7fa4b00cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:16.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.076+0000 7fa4b6200640 1 -- 172.21.15.49:0/1672894613 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b015f2b0 con 0x7fa4b006f320 2024-04-15T15:20:16.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.076+0000 7fa4b4e00640 1 --2- 172.21.15.49:0/1672894613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b015b860 0x7fa4b006ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.077+0000 7fa4b6200640 1 --2- 172.21.15.49:0/1672894613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b006f320 0x7fa4b00b3fc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009e50 tx=0x7fa4a802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ef7c6ed1e7cca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.077+0000 7fa4afe00640 1 -- 172.21.15.49:0/1672894613 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a802ca80 con 0x7fa4b006f320 2024-04-15T15:20:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.077+0000 7fa4afe00640 1 -- 172.21.15.49:0/1672894613 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a802cc20 con 0x7fa4b006f320 2024-04-15T15:20:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.077+0000 7fa4afe00640 1 -- 172.21.15.49:0/1672894613 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80343f0 con 0x7fa4b006f320 2024-04-15T15:20:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.078+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1672894613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b006f320 msgr2=0x7fa4b00b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.078+0000 7fa4b6c00640 1 --2- 172.21.15.49:0/1672894613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b006f320 0x7fa4b00b3fc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009e50 tx=0x7fa4a802bee0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.078+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1672894613 shutdown_connections 2024-04-15T15:20:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.078+0000 7fa4b6c00640 1 --2- 172.21.15.49:0/1672894613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b006f320 0x7fa4b00b3fc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.078+0000 7fa4b6c00640 1 --2- 172.21.15.49:0/1672894613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b015b860 0x7fa4b006ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.078+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1672894613 >> 172.21.15.49:0/1672894613 conn(0x7fa4b006b930 msgr2=0x7fa4b006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.078+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1672894613 shutdown_connections 2024-04-15T15:20:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.078+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1672894613 wait complete. 2024-04-15T15:20:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.080+0000 7fa4b6c00640 1 Processor -- start 2024-04-15T15:20:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.080+0000 7fa4b6c00640 1 -- start start 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b006f320 0x7fa4b013c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4b015b480 0x7fa4b0150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b015b860 0x7fa4b015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b015fb50 con 0x7fa4b015b860 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b015f870 con 0x7fa4b015b480 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b00cd550 con 0x7fa4b006f320 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b015b860 0x7fa4b015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b015b860 0x7fa4b015dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33086/0 (socket says 172.21.15.49:33086) 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6200640 1 -- 172.21.15.49:0/1117912325 learned_addr learned my addr 172.21.15.49:0/1117912325 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b5800640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b006f320 0x7fa4b013c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6200640 1 -- 172.21.15.49:0/1117912325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b006f320 msgr2=0x7fa4b013c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b4e00640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4b015b480 0x7fa4b0150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6200640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b006f320 0x7fa4b013c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6200640 1 -- 172.21.15.49:0/1117912325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4b015b480 msgr2=0x7fa4b0150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6200640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4b015b480 0x7fa4b0150510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.081+0000 7fa4b6200640 1 -- 172.21.15.49:0/1117912325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b013d3c0 con 0x7fa4b015b860 2024-04-15T15:20:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.082+0000 7fa4b6200640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b015b860 0x7fa4b015dde0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8036040 tx=0x7fa4a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.082+0000 7fa4ad600640 1 -- 172.21.15.49:0/1117912325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8043020 con 0x7fa4b015b860 2024-04-15T15:20:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.082+0000 7fa4ad600640 1 -- 172.21.15.49:0/1117912325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a803e450 con 0x7fa4b015b860 2024-04-15T15:20:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.082+0000 7fa4ad600640 1 -- 172.21.15.49:0/1117912325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a803e860 con 0x7fa4b015b860 2024-04-15T15:20:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.082+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1117912325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b013d6a0 con 0x7fa4b015b860 2024-04-15T15:20:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.082+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1117912325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b013dc60 con 0x7fa4b015b860 2024-04-15T15:20:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.083+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1117912325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b00755c0 con 0x7fa4b015b860 2024-04-15T15:20:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.084+0000 7fa4ad600640 1 -- 172.21.15.49:0/1117912325 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa4a8046020 con 0x7fa4b015b860 2024-04-15T15:20:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.085+0000 7fa4ad600640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa49006c9b0 0x7fa49006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.085+0000 7fa4ad600640 1 -- 172.21.15.49:0/1117912325 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa4a80ae340 con 0x7fa4b015b860 2024-04-15T15:20:16.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.085+0000 7fa4b5800640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa49006c9b0 0x7fa49006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.086+0000 7fa4b5800640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa49006c9b0 0x7fa49006ee70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4004ea0 tx=0x7fa4a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.090+0000 7fa4ad600640 1 -- 172.21.15.49:0/1117912325 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4a804d070 con 0x7fa4b015b860 2024-04-15T15:20:16.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.294+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1117912325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4b006bf20 con 0x7fa4b015b860 2024-04-15T15:20:16.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.296+0000 7fa4ad600640 1 -- 172.21.15.49:0/1117912325 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16669 (secure 0 0 0) 0x7fa4a807c5f0 con 0x7fa4b015b860 2024-04-15T15:20:16.298 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:16.298 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:01.591001+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:16.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.298+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1117912325 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa49006c9b0 msgr2=0x7fa49006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.298+0000 7fa4b6c00640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa49006c9b0 0x7fa49006ee70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4004ea0 tx=0x7fa4a4009040 comp rx=0 tx=0).stop 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.298+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1117912325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b015b860 msgr2=0x7fa4b015dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.298+0000 7fa4b6c00640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b015b860 0x7fa4b015dde0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8036040 tx=0x7fa4a8004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.298+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1117912325 shutdown_connections 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.298+0000 7fa4b6c00640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa49006c9b0 0x7fa49006ee70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.298+0000 7fa4b6c00640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4b015b860 0x7fa4b015dde0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.298+0000 7fa4b6c00640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4b015b480 0x7fa4b0150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.298+0000 7fa4b6c00640 1 --2- 172.21.15.49:0/1117912325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4b006f320 0x7fa4b013c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.298+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1117912325 >> 172.21.15.49:0/1117912325 conn(0x7fa4b006b930 msgr2=0x7fa4b006e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.299+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1117912325 shutdown_connections 2024-04-15T15:20:16.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.299+0000 7fa4b6c00640 1 -- 172.21.15.49:0/1117912325 wait complete. 2024-04-15T15:20:16.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:58.063498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:59.009342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:16.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:16.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.541+0000 7fa0cf200640 1 Processor -- start 2024-04-15T15:20:16.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.541+0000 7fa0cf200640 1 -- start start 2024-04-15T15:20:16.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.542+0000 7fa0cf200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0d015acc0 0x7fa0d00cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0cf200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa0d015ef00 0x7fa0d006e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0cf200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0d015b0a0 con 0x7fa0d00cfd40 2024-04-15T15:20:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0cf200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d00b5c50 con 0x7fa0d015ef00 2024-04-15T15:20:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0cf200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d00b6790 con 0x7fa0d015acc0 2024-04-15T15:20:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0ce800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa0d015ef00 0x7fa0d006e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0ce800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa0d015ef00 0x7fa0d006e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33090/0 (socket says 172.21.15.49:33090) 2024-04-15T15:20:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0ce800640 1 -- 172.21.15.49:0/1162869056 learned_addr learned my addr 172.21.15.49:0/1162869056 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0cde00640 1 --2- 172.21.15.49:0/1162869056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0d015acc0 0x7fa0d00cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0ce800640 1 -- 172.21.15.49:0/1162869056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0d015acc0 msgr2=0x7fa0d00cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:16.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0ce800640 1 --2- 172.21.15.49:0/1162869056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0d015acc0 0x7fa0d00cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.543+0000 7fa0ce800640 1 -- 172.21.15.49:0/1162869056 >> v1:172.21.15.107:6789/0 conn(0x7fa0d00cfd40 legacy=0x7fa0d015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:16.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.544+0000 7fa0ce800640 1 -- 172.21.15.49:0/1162869056 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d00b6210 con 0x7fa0d015ef00 2024-04-15T15:20:16.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.545+0000 7fa0ce800640 1 --2- 172.21.15.49:0/1162869056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015ef00 0x7fa0d006e790 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009e50 tx=0x7fa0c802e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367d26cb4b081484 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:16.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.545+0000 7fa0cca00640 1 -- 172.21.15.49:0/1162869056 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c802ec00 con 0x7fa0d015ef00 2024-04-15T15:20:16.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.545+0000 7fa0cca00640 1 -- 172.21.15.49:0/1162869056 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c802eda0 con 0x7fa0d015ef00 2024-04-15T15:20:16.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.545+0000 7fa0cca00640 1 -- 172.21.15.49:0/1162869056 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8036610 con 0x7fa0d015ef00 2024-04-15T15:20:16.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.546+0000 7fa0cf200640 1 -- 172.21.15.49:0/1162869056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015ef00 msgr2=0x7fa0d006e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:16.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.546+0000 7fa0cf200640 1 --2- 172.21.15.49:0/1162869056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015ef00 0x7fa0d006e790 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009e50 tx=0x7fa0c802e060 comp rx=0 tx=0).stop 2024-04-15T15:20:16.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.546+0000 7fa0cf200640 1 -- 172.21.15.49:0/1162869056 shutdown_connections 2024-04-15T15:20:16.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.546+0000 7fa0cf200640 1 --2- 172.21.15.49:0/1162869056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015ef00 0x7fa0d006e790 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.546+0000 7fa0cf200640 1 --2- 172.21.15.49:0/1162869056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0d015acc0 0x7fa0d00cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.546+0000 7fa0cf200640 1 -- 172.21.15.49:0/1162869056 >> 172.21.15.49:0/1162869056 conn(0x7fa0d00b23a0 msgr2=0x7fa0d00b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:16.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.546+0000 7fa0cf200640 1 -- 172.21.15.49:0/1162869056 shutdown_connections 2024-04-15T15:20:16.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.546+0000 7fa0cf200640 1 -- 172.21.15.49:0/1162869056 wait complete. 2024-04-15T15:20:16.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.548+0000 7fa0cf200640 1 Processor -- start 2024-04-15T15:20:16.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.548+0000 7fa0cf200640 1 -- start start 2024-04-15T15:20:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cf200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d00cfd40 0x7fa0d014a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cf200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015acc0 0x7fa0d00aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cf200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0d015ef00 0x7fa0d014ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cf200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d00b64b0 con 0x7fa0d015acc0 2024-04-15T15:20:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cf200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d00cf8b0 con 0x7fa0d00cfd40 2024-04-15T15:20:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cf200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d015f390 con 0x7fa0d015ef00 2024-04-15T15:20:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cd400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015acc0 0x7fa0d00aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cd400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015acc0 0x7fa0d00aeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33100/0 (socket says 172.21.15.49:33100) 2024-04-15T15:20:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cd400640 1 -- 172.21.15.49:0/2568137490 learned_addr learned my addr 172.21.15.49:0/2568137490 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:16.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cde00640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d00cfd40 0x7fa0d014a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0ce800640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0d015ef00 0x7fa0d014ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cd400640 1 -- 172.21.15.49:0/2568137490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0d015ef00 msgr2=0x7fa0d014ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:16.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.549+0000 7fa0cd400640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0d015ef00 0x7fa0d014ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.550+0000 7fa0cd400640 1 -- 172.21.15.49:0/2568137490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d00cfd40 msgr2=0x7fa0d014a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:16.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.550+0000 7fa0cd400640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d00cfd40 0x7fa0d014a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.550+0000 7fa0cd400640 1 -- 172.21.15.49:0/2568137490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d006f560 con 0x7fa0d015acc0 2024-04-15T15:20:16.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.550+0000 7fa0cd400640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015acc0 0x7fa0d00aeab0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00c400 tx=0x7fa0bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:16.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.550+0000 7fa0c6000640 1 -- 172.21.15.49:0/2568137490 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc011790 con 0x7fa0d015acc0 2024-04-15T15:20:16.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.550+0000 7fa0c6000640 1 -- 172.21.15.49:0/2568137490 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0bc017070 con 0x7fa0d015acc0 2024-04-15T15:20:16.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.550+0000 7fa0c6000640 1 -- 172.21.15.49:0/2568137490 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc011bb0 con 0x7fa0d015acc0 2024-04-15T15:20:16.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.550+0000 7fa0cf200640 1 -- 172.21.15.49:0/2568137490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d014a6f0 con 0x7fa0d015acc0 2024-04-15T15:20:16.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.550+0000 7fa0cf200640 1 -- 172.21.15.49:0/2568137490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d0147f00 con 0x7fa0d015acc0 2024-04-15T15:20:16.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.551+0000 7fa0cf200640 1 -- 172.21.15.49:0/2568137490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d00755c0 con 0x7fa0d015acc0 2024-04-15T15:20:16.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.552+0000 7fa0c6000640 1 -- 172.21.15.49:0/2568137490 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa0bc01c460 con 0x7fa0d015acc0 2024-04-15T15:20:16.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.553+0000 7fa0c6000640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa0b006c980 0x7fa0b006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:16.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.554+0000 7fa0c6000640 1 -- 172.21.15.49:0/2568137490 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa0bc08bae0 con 0x7fa0d015acc0 2024-04-15T15:20:16.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.554+0000 7fa0cde00640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa0b006c980 0x7fa0b006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:16.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.554+0000 7fa0cde00640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa0b006c980 0x7fa0b006ee40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0006fd0 tx=0x7fa0c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:16.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.558+0000 7fa0c6000640 1 -- 172.21.15.49:0/2568137490 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0d006f560 con 0x7fa0d015acc0 2024-04-15T15:20:16.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.770+0000 7fa0cf200640 1 -- 172.21.15.49:0/2568137490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0d00cd290 con 0x7fa0d015acc0 2024-04-15T15:20:16.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.771+0000 7fa0c6000640 1 -- 172.21.15.49:0/2568137490 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7fa0bc059e20 con 0x7fa0d015acc0 2024-04-15T15:20:16.772 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:20:16.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.773+0000 7fa0cf200640 1 -- 172.21.15.49:0/2568137490 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa0b006c980 msgr2=0x7fa0b006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.773+0000 7fa0cf200640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa0b006c980 0x7fa0b006ee40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0006fd0 tx=0x7fa0c0008040 comp rx=0 tx=0).stop 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.773+0000 7fa0cf200640 1 -- 172.21.15.49:0/2568137490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015acc0 msgr2=0x7fa0d00aeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.773+0000 7fa0cf200640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015acc0 0x7fa0d00aeab0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00c400 tx=0x7fa0bc00c8c0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.773+0000 7fa0cf200640 1 -- 172.21.15.49:0/2568137490 shutdown_connections 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.773+0000 7fa0cf200640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fa0b006c980 0x7fa0b006ee40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.773+0000 7fa0cf200640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0d015ef00 0x7fa0d014ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.773+0000 7fa0cf200640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d015acc0 0x7fa0d00aeab0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.773+0000 7fa0cf200640 1 --2- 172.21.15.49:0/2568137490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d00cfd40 0x7fa0d014a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.773+0000 7fa0cf200640 1 -- 172.21.15.49:0/2568137490 >> 172.21.15.49:0/2568137490 conn(0x7fa0d00b23a0 msgr2=0x7fa0d0159230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.774+0000 7fa0cf200640 1 -- 172.21.15.49:0/2568137490 shutdown_connections 2024-04-15T15:20:16.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:16.774+0000 7fa0cf200640 1 -- 172.21.15.49:0/2568137490 wait complete. 2024-04-15T15:20:16.795 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.026+0000 7fe3a7200640 1 Processor -- start 2024-04-15T15:20:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.026+0000 7fe3a7200640 1 -- start start 2024-04-15T15:20:17.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.028+0000 7fe3a7200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe3a806f740 0x7fe3a80cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.028+0000 7fe3a7200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe3a80b3dc0 0x7fe3a80cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.028+0000 7fe3a7200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a815f230 con 0x7fe3a8158f10 2024-04-15T15:20:17.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.028+0000 7fe3a7200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a815f4d0 con 0x7fe3a806f740 2024-04-15T15:20:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.028+0000 7fe3a7200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a815f7b0 con 0x7fe3a80b3dc0 2024-04-15T15:20:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.028+0000 7fe3a5400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe3a806f740 0x7fe3a80cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.028+0000 7fe3a5400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe3a806f740 0x7fe3a80cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33106/0 (socket says 172.21.15.49:33106) 2024-04-15T15:20:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.028+0000 7fe3a5400640 1 -- 172.21.15.49:0/3134784205 learned_addr learned my addr 172.21.15.49:0/3134784205 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.029+0000 7fe3a6800640 1 --2- 172.21.15.49:0/3134784205 >> v2:172.21.15.107:3300/0 conn(0x7fe3a80b3dc0 0x7fe3a80cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.029+0000 7fe3a5400640 1 -- 172.21.15.49:0/3134784205 >> v2:172.21.15.107:3300/0 conn(0x7fe3a80b3dc0 msgr2=0x7fe3a80cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.029+0000 7fe3a5400640 1 --2- 172.21.15.49:0/3134784205 >> v2:172.21.15.107:3300/0 conn(0x7fe3a80b3dc0 0x7fe3a80cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.029+0000 7fe3a5400640 1 -- 172.21.15.49:0/3134784205 >> v1:172.21.15.107:6789/0 conn(0x7fe3a8158f10 legacy=0x7fe3a806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.029+0000 7fe3a5400640 1 -- 172.21.15.49:0/3134784205 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a815fa90 con 0x7fe3a806f740 2024-04-15T15:20:17.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.030+0000 7fe3a5400640 1 --2- 172.21.15.49:0/3134784205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 0x7fe3a80cd4e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe3a806fc20 tx=0x7fe39802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5997dbcc1b60911d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:17.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.030+0000 7fe3a4a00640 1 -- 172.21.15.49:0/3134784205 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39802c9f0 con 0x7fe3a806f740 2024-04-15T15:20:17.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.030+0000 7fe3a4a00640 1 -- 172.21.15.49:0/3134784205 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39802cb90 con 0x7fe3a806f740 2024-04-15T15:20:17.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.030+0000 7fe3a4a00640 1 -- 172.21.15.49:0/3134784205 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3980343f0 con 0x7fe3a806f740 2024-04-15T15:20:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.031+0000 7fe3a7200640 1 -- 172.21.15.49:0/3134784205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 msgr2=0x7fe3a80cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.031+0000 7fe3a7200640 1 --2- 172.21.15.49:0/3134784205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 0x7fe3a80cd4e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe3a806fc20 tx=0x7fe39802be50 comp rx=0 tx=0).stop 2024-04-15T15:20:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.031+0000 7fe3a7200640 1 -- 172.21.15.49:0/3134784205 shutdown_connections 2024-04-15T15:20:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.031+0000 7fe3a7200640 1 --2- 172.21.15.49:0/3134784205 >> v2:172.21.15.107:3300/0 conn(0x7fe3a80b3dc0 0x7fe3a80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.031+0000 7fe3a7200640 1 --2- 172.21.15.49:0/3134784205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 0x7fe3a80cd4e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.031+0000 7fe3a7200640 1 -- 172.21.15.49:0/3134784205 >> 172.21.15.49:0/3134784205 conn(0x7fe3a806b930 msgr2=0x7fe3a806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.031+0000 7fe3a7200640 1 -- 172.21.15.49:0/3134784205 shutdown_connections 2024-04-15T15:20:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.031+0000 7fe3a7200640 1 -- 172.21.15.49:0/3134784205 wait complete. 2024-04-15T15:20:17.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.033+0000 7fe3a7200640 1 Processor -- start 2024-04-15T15:20:17.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.033+0000 7fe3a7200640 1 -- start start 2024-04-15T15:20:17.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.034+0000 7fe3a7200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 0x7fe3a814c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.034+0000 7fe3a7200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3a80b3dc0 0x7fe3a8143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.034+0000 7fe3a7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8158f10 0x7fe3a815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.034+0000 7fe3a7200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81592f0 con 0x7fe3a806f740 2024-04-15T15:20:17.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.034+0000 7fe3a7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a80cdf60 con 0x7fe3a8158f10 2024-04-15T15:20:17.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.034+0000 7fe3a7200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a8159470 con 0x7fe3a80b3dc0 2024-04-15T15:20:17.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.035+0000 7fe3a5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 0x7fe3a814c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.035+0000 7fe3a5e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 0x7fe3a814c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33112/0 (socket says 172.21.15.49:33112) 2024-04-15T15:20:17.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.035+0000 7fe3a5e00640 1 -- 172.21.15.49:0/447760652 learned_addr learned my addr 172.21.15.49:0/447760652 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:17.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.035+0000 7fe3a5400640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3a80b3dc0 0x7fe3a8143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.035+0000 7fe3a6800640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8158f10 0x7fe3a815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.035+0000 7fe3a5e00640 1 -- 172.21.15.49:0/447760652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3a80b3dc0 msgr2=0x7fe3a8143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.035+0000 7fe3a5e00640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3a80b3dc0 0x7fe3a8143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.035+0000 7fe3a5e00640 1 -- 172.21.15.49:0/447760652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8158f10 msgr2=0x7fe3a815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.035+0000 7fe3a5e00640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8158f10 0x7fe3a815a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.035+0000 7fe3a5e00640 1 -- 172.21.15.49:0/447760652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a815b140 con 0x7fe3a806f740 2024-04-15T15:20:17.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.036+0000 7fe3a5e00640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 0x7fe3a814c5e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe39000b3f0 tx=0x7fe39000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:17.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.036+0000 7fe39e000640 1 -- 172.21.15.49:0/447760652 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390015020 con 0x7fe3a806f740 2024-04-15T15:20:17.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.036+0000 7fe39e000640 1 -- 172.21.15.49:0/447760652 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe390011830 con 0x7fe3a806f740 2024-04-15T15:20:17.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.036+0000 7fe39e000640 1 -- 172.21.15.49:0/447760652 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390011c20 con 0x7fe3a806f740 2024-04-15T15:20:17.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.036+0000 7fe3a7200640 1 -- 172.21.15.49:0/447760652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a814bb00 con 0x7fe3a806f740 2024-04-15T15:20:17.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.036+0000 7fe3a7200640 1 -- 172.21.15.49:0/447760652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a814b350 con 0x7fe3a806f740 2024-04-15T15:20:17.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.037+0000 7fe3a7200640 1 -- 172.21.15.49:0/447760652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a80755c0 con 0x7fe3a806f740 2024-04-15T15:20:17.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.038+0000 7fe39e000640 1 -- 172.21.15.49:0/447760652 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe390018020 con 0x7fe3a806f740 2024-04-15T15:20:17.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.039+0000 7fe39e000640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe38406c980 0x7fe38406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.039+0000 7fe39e000640 1 -- 172.21.15.49:0/447760652 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe39008b800 con 0x7fe3a806f740 2024-04-15T15:20:17.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.039+0000 7fe3a5400640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe38406c980 0x7fe38406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.040+0000 7fe3a5400640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe38406c980 0x7fe38406ee40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe398035000 tx=0x7fe398030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:17.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.043+0000 7fe39e000640 1 -- 172.21.15.49:0/447760652 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe39001f030 con 0x7fe3a806f740 2024-04-15T15:20:17.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.250+0000 7fe3a7200640 1 -- 172.21.15.49:0/447760652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3a806cf70 con 0x7fe3a806f740 2024-04-15T15:20:17.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.251+0000 7fe39e000640 1 -- 172.21.15.49:0/447760652 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7fe390059c40 con 0x7fe3a806f740 2024-04-15T15:20:17.253 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:20:17.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.253+0000 7fe3a7200640 1 -- 172.21.15.49:0/447760652 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe38406c980 msgr2=0x7fe38406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe38406c980 0x7fe38406ee40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe398035000 tx=0x7fe398030000 comp rx=0 tx=0).stop 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 -- 172.21.15.49:0/447760652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 msgr2=0x7fe3a814c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 0x7fe3a814c5e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe39000b3f0 tx=0x7fe39000b8b0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 -- 172.21.15.49:0/447760652 shutdown_connections 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe38406c980 0x7fe38406ee40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3a80b3dc0 0x7fe3a8143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8158f10 0x7fe3a815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 --2- 172.21.15.49:0/447760652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a806f740 0x7fe3a814c5e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 -- 172.21.15.49:0/447760652 >> 172.21.15.49:0/447760652 conn(0x7fe3a806b930 msgr2=0x7fe3a80b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 -- 172.21.15.49:0/447760652 shutdown_connections 2024-04-15T15:20:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.254+0000 7fe3a7200640 1 -- 172.21.15.49:0/447760652 wait complete. 2024-04-15T15:20:17.276 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:17.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.505+0000 7ff5e6200640 1 Processor -- start 2024-04-15T15:20:17.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.506+0000 7ff5e6200640 1 -- start start 2024-04-15T15:20:17.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.507+0000 7ff5e6200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff5e01461c0 0x7ff5e0071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.507+0000 7ff5e6200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff5e0072420 0x7ff5e00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.507+0000 7ff5e6200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e00cd2d0 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.507+0000 7ff5e6200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e00cf1d0 con 0x7ff5e01461c0 2024-04-15T15:20:17.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.507+0000 7ff5e6200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e0072880 con 0x7ff5e0072420 2024-04-15T15:20:17.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.508+0000 7ff5e5800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff5e00ce9e0 0x7ff5e0160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55106/0 (socket says 172.21.15.49:55106) 2024-04-15T15:20:17.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.508+0000 7ff5e5800640 1 -- 172.21.15.49:0/2118298425 learned_addr learned my addr 172.21.15.49:0/2118298425 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:17.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.508+0000 7ff5e4e00640 1 --2- 172.21.15.49:0/2118298425 >> v2:172.21.15.49:3300/0 conn(0x7ff5e01461c0 0x7ff5e0071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.508+0000 7ff5dfe00640 1 --2- 172.21.15.49:0/2118298425 >> v2:172.21.15.107:3300/0 conn(0x7ff5e0072420 0x7ff5e00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.508+0000 7ff5df400640 1 -- 172.21.15.49:0/2118298425 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3366827402 0 0) 0x7ff5e00cd2d0 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.508+0000 7ff5e4e00640 1 -- 172.21.15.49:0/2118298425 >> v2:172.21.15.107:3300/0 conn(0x7ff5e0072420 msgr2=0x7ff5e00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:17.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.508+0000 7ff5e4e00640 1 --2- 172.21.15.49:0/2118298425 >> v2:172.21.15.107:3300/0 conn(0x7ff5e0072420 0x7ff5e00ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.508+0000 7ff5e4e00640 1 -- 172.21.15.49:0/2118298425 >> v1:172.21.15.49:6789/0 conn(0x7ff5e00ce9e0 legacy=0x7ff5e0160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.508+0000 7ff5e4e00640 1 -- 172.21.15.49:0/2118298425 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e00cd5b0 con 0x7ff5e01461c0 2024-04-15T15:20:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.510+0000 7ff5e4e00640 1 --2- 172.21.15.49:0/2118298425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e01461c0 0x7ff5e0071ee0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00737f0 tx=0x7ff5d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503041285f4be80f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.510+0000 7ff5df400640 1 -- 172.21.15.49:0/2118298425 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4035020 con 0x7ff5e01461c0 2024-04-15T15:20:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.510+0000 7ff5df400640 1 -- 172.21.15.49:0/2118298425 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d4031830 con 0x7ff5e01461c0 2024-04-15T15:20:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.510+0000 7ff5df400640 1 -- 172.21.15.49:0/2118298425 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4031c00 con 0x7ff5e01461c0 2024-04-15T15:20:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.511+0000 7ff5e6200640 1 -- 172.21.15.49:0/2118298425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e01461c0 msgr2=0x7ff5e0071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.511+0000 7ff5e6200640 1 --2- 172.21.15.49:0/2118298425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e01461c0 0x7ff5e0071ee0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00737f0 tx=0x7ff5d402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.511+0000 7ff5e6200640 1 -- 172.21.15.49:0/2118298425 shutdown_connections 2024-04-15T15:20:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.511+0000 7ff5e6200640 1 --2- 172.21.15.49:0/2118298425 >> v2:172.21.15.107:3300/0 conn(0x7ff5e0072420 0x7ff5e00ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.511+0000 7ff5e6200640 1 --2- 172.21.15.49:0/2118298425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e01461c0 0x7ff5e0071ee0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.511+0000 7ff5e6200640 1 -- 172.21.15.49:0/2118298425 >> 172.21.15.49:0/2118298425 conn(0x7ff5e006b930 msgr2=0x7ff5e006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.511+0000 7ff5e6200640 1 -- 172.21.15.49:0/2118298425 shutdown_connections 2024-04-15T15:20:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.511+0000 7ff5e6200640 1 -- 172.21.15.49:0/2118298425 wait complete. 2024-04-15T15:20:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.513+0000 7ff5e6200640 1 Processor -- start 2024-04-15T15:20:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.513+0000 7ff5e6200640 1 -- start start 2024-04-15T15:20:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.514+0000 7ff5e6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e0072420 0x7ff5e01565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.514+0000 7ff5e6200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e00ce9e0 0x7ff5e015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.514+0000 7ff5e6200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e01461c0 0x7ff5e0135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.514+0000 7ff5e6200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e00cdb70 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.514+0000 7ff5e6200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e00cd080 con 0x7ff5e0072420 2024-04-15T15:20:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.514+0000 7ff5e6200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e00cee50 con 0x7ff5e01461c0 2024-04-15T15:20:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.514+0000 7ff5dfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e00ce9e0 0x7ff5e015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.514+0000 7ff5dfe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e00ce9e0 0x7ff5e015ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33124/0 (socket says 172.21.15.49:33124) 2024-04-15T15:20:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.514+0000 7ff5dfe00640 1 -- 172.21.15.49:0/3041149927 learned_addr learned my addr 172.21.15.49:0/3041149927 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.515+0000 7ff5e5800640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e01461c0 0x7ff5e0135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.515+0000 7ff5e4e00640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e0072420 0x7ff5e01565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.515+0000 7ff5dfe00640 1 -- 172.21.15.49:0/3041149927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e01461c0 msgr2=0x7ff5e0135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.515+0000 7ff5dfe00640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e01461c0 0x7ff5e0135640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.515+0000 7ff5dfe00640 1 -- 172.21.15.49:0/3041149927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e0072420 msgr2=0x7ff5e01565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.515+0000 7ff5dfe00640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e0072420 0x7ff5e01565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.515+0000 7ff5dfe00640 1 -- 172.21.15.49:0/3041149927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e0159b80 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.515+0000 7ff5e4e00640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e0072420 0x7ff5e01565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.515+0000 7ff5dfe00640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e00ce9e0 0x7ff5e015ac20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00d800 tx=0x7ff5cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.515+0000 7ff5dcc00640 1 -- 172.21.15.49:0/3041149927 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00b6c0 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.516+0000 7ff5dcc00640 1 -- 172.21.15.49:0/3041149927 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5cc019070 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.516+0000 7ff5e6200640 1 -- 172.21.15.49:0/3041149927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e0159e60 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.516+0000 7ff5dcc00640 1 -- 172.21.15.49:0/3041149927 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00ba80 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.516+0000 7ff5e6200640 1 -- 172.21.15.49:0/3041149927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e015a420 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.517+0000 7ff5e6200640 1 -- 172.21.15.49:0/3041149927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e00755c0 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.518+0000 7ff5dcc00640 1 -- 172.21.15.49:0/3041149927 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff5cc00bc20 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.519+0000 7ff5dcc00640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff5c006c9d0 0x7ff5c006ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.519+0000 7ff5dcc00640 1 -- 172.21.15.49:0/3041149927 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7ff5cc08bfe0 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.519+0000 7ff5e4e00640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff5c006c9d0 0x7ff5c006ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.520+0000 7ff5e4e00640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff5c006c9d0 0x7ff5c006ee90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4036000 tx=0x7ff5d4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:17.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.524+0000 7ff5dcc00640 1 -- 172.21.15.49:0/3041149927 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5cc05a4a0 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.729+0000 7ff5e6200640 1 -- 172.21.15.49:0/3041149927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff5e0072e40 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.729+0000 7ff5dcc00640 1 -- 172.21.15.49:0/3041149927 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7ff5cc059da0 con 0x7ff5e00ce9e0 2024-04-15T15:20:17.731 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:20:17.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.732+0000 7ff5e6200640 1 -- 172.21.15.49:0/3041149927 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff5c006c9d0 msgr2=0x7ff5c006ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.732+0000 7ff5e6200640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff5c006c9d0 0x7ff5c006ee90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4036000 tx=0x7ff5d4008030 comp rx=0 tx=0).stop 2024-04-15T15:20:17.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.732+0000 7ff5e6200640 1 -- 172.21.15.49:0/3041149927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e00ce9e0 msgr2=0x7ff5e015ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.732+0000 7ff5e6200640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e00ce9e0 0x7ff5e015ac20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00d800 tx=0x7ff5cc00dcc0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.732+0000 7ff5e6200640 1 -- 172.21.15.49:0/3041149927 shutdown_connections 2024-04-15T15:20:17.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.732+0000 7ff5e6200640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff5c006c9d0 0x7ff5c006ee90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.732+0000 7ff5e6200640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e01461c0 0x7ff5e0135640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.732+0000 7ff5e6200640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e00ce9e0 0x7ff5e015ac20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.732+0000 7ff5e6200640 1 --2- 172.21.15.49:0/3041149927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e0072420 0x7ff5e01565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.732+0000 7ff5e6200640 1 -- 172.21.15.49:0/3041149927 >> 172.21.15.49:0/3041149927 conn(0x7ff5e006b930 msgr2=0x7ff5e00b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:17.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.733+0000 7ff5e6200640 1 -- 172.21.15.49:0/3041149927 shutdown_connections 2024-04-15T15:20:17.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.733+0000 7ff5e6200640 1 -- 172.21.15.49:0/3041149927 wait complete. 2024-04-15T15:20:17.753 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:17.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.970+0000 7ffb1a600640 1 Processor -- start 2024-04-15T15:20:17.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.970+0000 7ffb1a600640 1 -- start start 2024-04-15T15:20:17.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.971+0000 7ffb1a600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffb1415eab0 0x7ffb14162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.971+0000 7ffb1a600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 0x7ffb140d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.971+0000 7ffb1a600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb140b78f0 con 0x7ffb14162b70 2024-04-15T15:20:17.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.971+0000 7ffb1a600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ffb140d2e50 con 0x7ffb1415eab0 2024-04-15T15:20:17.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.971+0000 7ffb1a600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ffb14163010 con 0x7ffb140d1c60 2024-04-15T15:20:17.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.971+0000 7ffb19200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffb1415eab0 0x7ffb14162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.972+0000 7ffb19200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffb1415eab0 0x7ffb14162630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33132/0 (socket says 172.21.15.49:33132) 2024-04-15T15:20:17.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.972+0000 7ffb19200640 1 -- 172.21.15.49:0/2178143681 learned_addr learned my addr 172.21.15.49:0/2178143681 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:17.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.972+0000 7ffb19c00640 1 --2- 172.21.15.49:0/2178143681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 0x7ffb140d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.972+0000 7ffb19200640 1 -- 172.21.15.49:0/2178143681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 msgr2=0x7ffb140d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:17.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.972+0000 7ffb19200640 1 --2- 172.21.15.49:0/2178143681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 0x7ffb140d32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.972+0000 7ffb19200640 1 -- 172.21.15.49:0/2178143681 >> v1:172.21.15.107:6789/0 conn(0x7ffb14162b70 legacy=0x7ffb140d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:17.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.972+0000 7ffb19200640 1 -- 172.21.15.49:0/2178143681 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb141632f0 con 0x7ffb1415eab0 2024-04-15T15:20:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.974+0000 7ffb19200640 1 --2- 172.21.15.49:0/2178143681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 0x7ffb14162630 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009870 tx=0x7ffb0402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3693271e938ea598 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.974+0000 7ffb13400640 1 -- 172.21.15.49:0/2178143681 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04035020 con 0x7ffb1415eab0 2024-04-15T15:20:17.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.974+0000 7ffb13400640 1 -- 172.21.15.49:0/2178143681 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb04031640 con 0x7ffb1415eab0 2024-04-15T15:20:17.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.974+0000 7ffb13400640 1 -- 172.21.15.49:0/2178143681 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0402d040 con 0x7ffb1415eab0 2024-04-15T15:20:17.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.975+0000 7ffb1a600640 1 -- 172.21.15.49:0/2178143681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 msgr2=0x7ffb14162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.975+0000 7ffb1a600640 1 --2- 172.21.15.49:0/2178143681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 0x7ffb14162630 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009870 tx=0x7ffb0402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.975+0000 7ffb1a600640 1 -- 172.21.15.49:0/2178143681 shutdown_connections 2024-04-15T15:20:17.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.975+0000 7ffb1a600640 1 --2- 172.21.15.49:0/2178143681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 0x7ffb140d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.975+0000 7ffb1a600640 1 --2- 172.21.15.49:0/2178143681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 0x7ffb14162630 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.975+0000 7ffb1a600640 1 -- 172.21.15.49:0/2178143681 >> 172.21.15.49:0/2178143681 conn(0x7ffb14073240 msgr2=0x7ffb14073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:17.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.975+0000 7ffb1a600640 1 -- 172.21.15.49:0/2178143681 shutdown_connections 2024-04-15T15:20:17.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.975+0000 7ffb1a600640 1 -- 172.21.15.49:0/2178143681 wait complete. 2024-04-15T15:20:17.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.977+0000 7ffb1a600640 1 Processor -- start 2024-04-15T15:20:17.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.977+0000 7ffb1a600640 1 -- start start 2024-04-15T15:20:17.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.978+0000 7ffb1a600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 0x7ffb1415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.978+0000 7ffb1a600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 0x7ffb14148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.978+0000 7ffb1a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb14162b70 0x7ffb14149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.978+0000 7ffb1a600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffb140ba590 con 0x7ffb1415eab0 2024-04-15T15:20:17.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.978+0000 7ffb1a600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1415ee90 con 0x7ffb14162b70 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.978+0000 7ffb1a600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ffb140ba710 con 0x7ffb140d1c60 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.978+0000 7ffb13e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 0x7ffb14148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.978+0000 7ffb13e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 0x7ffb14148dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33146/0 (socket says 172.21.15.49:33146) 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.978+0000 7ffb13e00640 1 -- 172.21.15.49:0/768520050 learned_addr learned my addr 172.21.15.49:0/768520050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb19c00640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb14162b70 0x7ffb14149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb19200640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 0x7ffb1415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb13e00640 1 -- 172.21.15.49:0/768520050 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 msgr2=0x7ffb1415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb13e00640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 0x7ffb1415f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb13e00640 1 -- 172.21.15.49:0/768520050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb14162b70 msgr2=0x7ffb14149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb13e00640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb14162b70 0x7ffb14149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb13e00640 1 -- 172.21.15.49:0/768520050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1415c810 con 0x7ffb1415eab0 2024-04-15T15:20:17.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb19200640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 0x7ffb1415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:17.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb13e00640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 0x7ffb14148dc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00b3f0 tx=0x7ffafc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:17.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb10c00640 1 -- 172.21.15.49:0/768520050 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc015020 con 0x7ffb1415eab0 2024-04-15T15:20:17.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb10c00640 1 -- 172.21.15.49:0/768520050 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffafc011830 con 0x7ffb1415eab0 2024-04-15T15:20:17.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.979+0000 7ffb10c00640 1 -- 172.21.15.49:0/768520050 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc011c20 con 0x7ffb1415eab0 2024-04-15T15:20:17.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.980+0000 7ffb1a600640 1 -- 172.21.15.49:0/768520050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1415caf0 con 0x7ffb1415eab0 2024-04-15T15:20:17.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.980+0000 7ffb1a600640 1 -- 172.21.15.49:0/768520050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1415d0b0 con 0x7ffb1415eab0 2024-04-15T15:20:17.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.981+0000 7ffb1a600640 1 -- 172.21.15.49:0/768520050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb140cb5e0 con 0x7ffb1415eab0 2024-04-15T15:20:17.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.982+0000 7ffb10c00640 1 -- 172.21.15.49:0/768520050 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ffafc018020 con 0x7ffb1415eab0 2024-04-15T15:20:17.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.983+0000 7ffb10c00640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ffaf006c980 0x7ffaf006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:17.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.983+0000 7ffb10c00640 1 -- 172.21.15.49:0/768520050 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7ffafc08bce0 con 0x7ffb1415eab0 2024-04-15T15:20:17.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.983+0000 7ffb19200640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ffaf006c980 0x7ffaf006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:17.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.984+0000 7ffb19200640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ffaf006c980 0x7ffaf006ee40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffb04036000 tx=0x7ffb04008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:17.987+0000 7ffb10c00640 1 -- 172.21.15.49:0/768520050 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb140cb5e0 con 0x7ffb1415eab0 2024-04-15T15:20:18.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.203+0000 7ffb1a600640 1 -- 172.21.15.49:0/768520050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffb140755c0 con 0x7ffb1415eab0 2024-04-15T15:20:18.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.204+0000 7ffb10c00640 1 -- 172.21.15.49:0/768520050 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7ffafc01f020 con 0x7ffb1415eab0 2024-04-15T15:20:18.205 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2024-04-15T15:20:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 -- 172.21.15.49:0/768520050 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ffaf006c980 msgr2=0x7ffaf006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ffaf006c980 0x7ffaf006ee40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffb04036000 tx=0x7ffb04008030 comp rx=0 tx=0).stop 2024-04-15T15:20:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 -- 172.21.15.49:0/768520050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 msgr2=0x7ffb14148dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 0x7ffb14148dc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00b3f0 tx=0x7ffafc00b8b0 comp rx=0 tx=0).stop 2024-04-15T15:20:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 -- 172.21.15.49:0/768520050 shutdown_connections 2024-04-15T15:20:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ffaf006c980 0x7ffaf006ee40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:18.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb14162b70 0x7ffb14149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:18.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb1415eab0 0x7ffb14148dc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:18.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 --2- 172.21.15.49:0/768520050 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb140d1c60 0x7ffb1415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:18.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 -- 172.21.15.49:0/768520050 >> 172.21.15.49:0/768520050 conn(0x7ffb14073240 msgr2=0x7ffb140c9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:18.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.206+0000 7ffb1a600640 1 -- 172.21.15.49:0/768520050 shutdown_connections 2024-04-15T15:20:18.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.207+0000 7ffb1a600640 1 -- 172.21.15.49:0/768520050 wait complete. 2024-04-15T15:20:18.228 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-15T15:20:18.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:18.229 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:18.339 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:20:18.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:18.340 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-15T15:20:18.392 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:20:18.392 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:18.454 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052756,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-15T15:20:18.455 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:20:18.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:18.455 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-15T15:20:18.511 INFO:teuthology.orchestra.run.smithi049.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-15T15:20:18.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:20:18.539 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:20:18.539 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:18.597 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-15T15:20:18.597 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:20:18.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:18.598 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-15T15:20:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-15T15:20:18.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:20:18.651 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.885+0000 7fdb2d600640 1 Processor -- start 2024-04-15T15:20:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.885+0000 7fdb2d600640 1 -- start start 2024-04-15T15:20:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.887+0000 7fdb2d600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdb2815d2f0 0x7fdb2806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.887+0000 7fdb2d600640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb2815dbb0 con 0x7fdb280cee00 2024-04-15T15:20:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.887+0000 7fdb2d600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb2815d8d0 con 0x7fdb28072660 2024-04-15T15:20:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.887+0000 7fdb2d600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fdb2806f4f0 con 0x7fdb2815d2f0 2024-04-15T15:20:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.887+0000 7fdb27e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdb280cee00 0x7fdb28071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55120/0 (socket says 172.21.15.49:55120) 2024-04-15T15:20:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.887+0000 7fdb27e00640 1 -- 172.21.15.49:0/1107811697 learned_addr learned my addr 172.21.15.49:0/1107811697 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.887+0000 7fdb2cc00640 1 --2- 172.21.15.49:0/1107811697 >> v2:172.21.15.49:3300/0 conn(0x7fdb2815d2f0 0x7fdb2806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.888+0000 7fdb2cc00640 1 -- 172.21.15.49:0/1107811697 >> v1:172.21.15.107:6789/0 conn(0x7fdb28072660 legacy=0x7fdb2815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.888+0000 7fdb2cc00640 1 -- 172.21.15.49:0/1107811697 >> v1:172.21.15.49:6789/0 conn(0x7fdb280cee00 legacy=0x7fdb28071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.888+0000 7fdb26a00640 1 -- 172.21.15.49:0/1107811697 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3124083384 0 0) 0x7fdb2815dbb0 con 0x7fdb280cee00 2024-04-15T15:20:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.888+0000 7fdb2cc00640 1 -- 172.21.15.49:0/1107811697 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2806dde0 con 0x7fdb2815d2f0 2024-04-15T15:20:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.889+0000 7fdb2cc00640 1 --2- 172.21.15.49:0/1107811697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb2815d2f0 0x7fdb2806e8c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009d90 tx=0x7fdb2002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359dc0b9b9399e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.889+0000 7fdb26a00640 1 -- 172.21.15.49:0/1107811697 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20039070 con 0x7fdb2815d2f0 2024-04-15T15:20:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.889+0000 7fdb26a00640 1 -- 172.21.15.49:0/1107811697 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2002cc60 con 0x7fdb2815d2f0 2024-04-15T15:20:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.889+0000 7fdb26a00640 1 -- 172.21.15.49:0/1107811697 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20034660 con 0x7fdb2815d2f0 2024-04-15T15:20:18.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.890+0000 7fdb2d600640 1 -- 172.21.15.49:0/1107811697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb2815d2f0 msgr2=0x7fdb2806e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:18.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.890+0000 7fdb2d600640 1 --2- 172.21.15.49:0/1107811697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb2815d2f0 0x7fdb2806e8c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009d90 tx=0x7fdb2002beb0 comp rx=0 tx=0).stop 2024-04-15T15:20:18.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.890+0000 7fdb2d600640 1 -- 172.21.15.49:0/1107811697 shutdown_connections 2024-04-15T15:20:18.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.890+0000 7fdb2d600640 1 --2- 172.21.15.49:0/1107811697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb2815d2f0 0x7fdb2806e8c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:18.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.890+0000 7fdb2d600640 1 -- 172.21.15.49:0/1107811697 >> 172.21.15.49:0/1107811697 conn(0x7fdb2806b930 msgr2=0x7fdb2806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:18.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.890+0000 7fdb2d600640 1 -- 172.21.15.49:0/1107811697 shutdown_connections 2024-04-15T15:20:18.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.890+0000 7fdb2d600640 1 -- 172.21.15.49:0/1107811697 wait complete. 2024-04-15T15:20:18.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.892+0000 7fdb2d600640 1 Processor -- start 2024-04-15T15:20:18.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.892+0000 7fdb2d600640 1 -- start start 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.893+0000 7fdb2d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb28072660 0x7fdb28158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.893+0000 7fdb2d600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb280cee00 0x7fdb280ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.893+0000 7fdb2d600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb2815d2f0 0x7fdb28160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.893+0000 7fdb2d600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2815dda0 con 0x7fdb280cee00 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.893+0000 7fdb2d600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2806e3a0 con 0x7fdb28072660 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.893+0000 7fdb2d600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2815df20 con 0x7fdb2815d2f0 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.893+0000 7fdb27400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb280cee00 0x7fdb280ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.893+0000 7fdb27400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb280cee00 0x7fdb280ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33158/0 (socket says 172.21.15.49:33158) 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.893+0000 7fdb27400640 1 -- 172.21.15.49:0/4244062400 learned_addr learned my addr 172.21.15.49:0/4244062400 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.894+0000 7fdb27e00640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb28072660 0x7fdb28158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.894+0000 7fdb2cc00640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb2815d2f0 0x7fdb28160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:18.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.894+0000 7fdb27400640 1 -- 172.21.15.49:0/4244062400 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb2815d2f0 msgr2=0x7fdb28160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:18.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.894+0000 7fdb27400640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb2815d2f0 0x7fdb28160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:18.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.894+0000 7fdb27400640 1 -- 172.21.15.49:0/4244062400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb28072660 msgr2=0x7fdb28158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:18.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.894+0000 7fdb27400640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb28072660 0x7fdb28158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:18.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.894+0000 7fdb27400640 1 -- 172.21.15.49:0/4244062400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb280c59e0 con 0x7fdb280cee00 2024-04-15T15:20:18.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.894+0000 7fdb27400640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb280cee00 0x7fdb280ca380 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400c430 tx=0x7fdb1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:18.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.895+0000 7fdb1fe00640 1 -- 172.21.15.49:0/4244062400 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14011790 con 0x7fdb280cee00 2024-04-15T15:20:18.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.895+0000 7fdb1fe00640 1 -- 172.21.15.49:0/4244062400 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb14017070 con 0x7fdb280cee00 2024-04-15T15:20:18.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.895+0000 7fdb1fe00640 1 -- 172.21.15.49:0/4244062400 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14011bb0 con 0x7fdb280cee00 2024-04-15T15:20:18.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.895+0000 7fdb2d600640 1 -- 172.21.15.49:0/4244062400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb280c5cc0 con 0x7fdb280cee00 2024-04-15T15:20:18.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.895+0000 7fdb2d600640 1 -- 172.21.15.49:0/4244062400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb280c6280 con 0x7fdb280cee00 2024-04-15T15:20:18.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.896+0000 7fdb2d600640 1 -- 172.21.15.49:0/4244062400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb280755c0 con 0x7fdb280cee00 2024-04-15T15:20:18.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.897+0000 7fdb1fe00640 1 -- 172.21.15.49:0/4244062400 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fdb1401c460 con 0x7fdb280cee00 2024-04-15T15:20:18.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.898+0000 7fdb1fe00640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdb0806c980 0x7fdb0806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:18.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.898+0000 7fdb1fe00640 1 -- 172.21.15.49:0/4244062400 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fdb1408baf0 con 0x7fdb280cee00 2024-04-15T15:20:18.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.898+0000 7fdb27e00640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdb0806c980 0x7fdb0806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:18.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.899+0000 7fdb27e00640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdb0806c980 0x7fdb0806ee40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb18007fd0 tx=0x7fdb18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:18.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:18.903+0000 7fdb1fe00640 1 -- 172.21.15.49:0/4244062400 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb14059ec0 con 0x7fdb280cee00 2024-04-15T15:20:19.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.153+0000 7fdb2d600640 1 -- 172.21.15.49:0/4244062400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdb281594b0 con 0x7fdb280cee00 2024-04-15T15:20:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.154+0000 7fdb1fe00640 1 -- 172.21.15.49:0/4244062400 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fdb140597c0 con 0x7fdb280cee00 2024-04-15T15:20:19.156 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 -- 172.21.15.49:0/4244062400 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdb0806c980 msgr2=0x7fdb0806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdb0806c980 0x7fdb0806ee40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb18007fd0 tx=0x7fdb18009040 comp rx=0 tx=0).stop 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 -- 172.21.15.49:0/4244062400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb280cee00 msgr2=0x7fdb280ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb280cee00 0x7fdb280ca380 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400c430 tx=0x7fdb1400c8f0 comp rx=0 tx=0).stop 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 -- 172.21.15.49:0/4244062400 shutdown_connections 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdb0806c980 0x7fdb0806ee40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb2815d2f0 0x7fdb28160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb280cee00 0x7fdb280ca380 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 --2- 172.21.15.49:0/4244062400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb28072660 0x7fdb28158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 -- 172.21.15.49:0/4244062400 >> 172.21.15.49:0/4244062400 conn(0x7fdb2806b930 msgr2=0x7fdb280af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 -- 172.21.15.49:0/4244062400 shutdown_connections 2024-04-15T15:20:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19.157+0000 7fdb2d600640 1 -- 172.21.15.49:0/4244062400 wait complete. 2024-04-15T15:20:19.177 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:20:19.177 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:19.177 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-04-15T15:20:19.177 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-15T15:20:19.177 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:19.177 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:20:19.177 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:19.177 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:19.177 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:19.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:19.178 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-04-15T15:20:19.234 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:19.235 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42483sec preferred_lft 42483sec 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 3a:d8:a3:bb:09:2d brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::6c47:44ff:feb2:eb9c/64 scope link 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:19.236 INFO:teuthology.orchestra.run.smithi049.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 3a:d8:a3:bb:09:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::38d8:a3ff:febb:92d/64 scope link 2024-04-15T15:20:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:19.237 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 66:4c:0b:61:1e:4f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::644c:bff:fe61:1e4f/64 scope link 2024-04-15T15:20:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:19.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:19.238 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:19.336 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-04-15T15:20:19.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:19.341 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-04-15T15:20:19.399 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-04-15T15:20:19.399 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:19.399 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-15T15:20:19.399 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-15T15:20:19.399 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:19.400 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:19.400 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:20:19.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-15T15:20:19.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-15T15:20:19.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-15T15:20:19.400 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:20:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:19.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:19.678 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-04-15T15:20:19.724 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:20:19.724 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:20:19.725 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:19.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:19.725 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-04-15T15:20:19.828 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:19.828 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:19.886 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:20:19.887 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:19.887 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:19.944 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:19.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:19.944 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:20:20.101 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:20:20.101 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:20:20.101 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:20:20.127 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:20:20.127 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:20:20.127 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:20:20.128 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:20.128 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:20.153 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:20:20.153 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:20:20.153 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:20:20.153 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:20.153 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:20.153 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790" 2024-04-15T15:20:20.153 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790:/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:20.153 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c" will pass to kernel 2024-04-15T15:20:20.155 INFO:teuthology.orchestra.run.smithi049.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-15T15:20:20.155 INFO:teuthology.orchestra.run.smithi049.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:20:20.155 INFO:teuthology.orchestra.run.smithi049.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:20:20.155 INFO:teuthology.orchestra.run.smithi049.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:20:20.155 INFO:teuthology.orchestra.run.smithi049.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:20:20.158 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:20:20.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:20.158 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:20.238 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:20:20.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:20:20.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:20.348 INFO:teuthology.orchestra.run.smithi049.stdout:client5361 2024-04-15T15:20:20.348 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:20.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.541+0000 7f17e4200640 1 Processor -- start 2024-04-15T15:20:20.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.541+0000 7f17e4200640 1 -- start start 2024-04-15T15:20:20.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.543+0000 7f17e4200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f17dc0b51f0 0x7f17dc15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:20.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.543+0000 7f17e4200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17dc06f940 con 0x7f17dc070210 2024-04-15T15:20:20.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.543+0000 7f17e4200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17dc06e8a0 con 0x7f17dc15b6d0 2024-04-15T15:20:20.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.543+0000 7f17e4200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f17dc06eb80 con 0x7f17dc0b51f0 2024-04-15T15:20:20.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.543+0000 7f17e0e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f17dc070210 0x7f17dc0b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55124/0 (socket says 172.21.15.49:55124) 2024-04-15T15:20:20.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.543+0000 7f17e0e00640 1 -- 172.21.15.49:0/2932837582 learned_addr learned my addr 172.21.15.49:0/2932837582 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:20.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.543+0000 7f17e2200640 1 --2- 172.21.15.49:0/2932837582 >> v2:172.21.15.107:3300/0 conn(0x7f17dc0b51f0 0x7f17dc15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 909608957 0 0) 0x7f17dc06f940 con 0x7f17dc070210 2024-04-15T15:20:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17c4003520 con 0x7f17dc070210 2024-04-15T15:20:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3598027422 0 0) 0x7f17c4003520 con 0x7f17dc070210 2024-04-15T15:20:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 >> v2:172.21.15.107:3300/0 conn(0x7f17dc0b51f0 msgr2=0x7f17dc15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 --2- 172.21.15.49:0/2932837582 >> v2:172.21.15.107:3300/0 conn(0x7f17dc0b51f0 0x7f17dc15cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 >> v1:172.21.15.107:6789/0 conn(0x7f17dc15b6d0 legacy=0x7f17dc0cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc06ee60 con 0x7f17dc070210 2024-04-15T15:20:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f17c8001670 con 0x7f17dc070210 2024-04-15T15:20:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 >> v1:172.21.15.49:6789/0 conn(0x7f17dc070210 legacy=0x7f17dc0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40039b0 0x7f17c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c4004800 0x7f17c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c40051c0 0x7f17c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc06eb80 con 0x7f17c40051c0 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17c400c160 con 0x7f17c40039b0 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f17c400e570 con 0x7f17c4004800 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.544+0000 7f17e2200640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c40051c0 0x7f17c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.545+0000 7f17e0e00640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c4004800 0x7f17c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.545+0000 7f17e2200640 1 -- 172.21.15.49:0/2932837582 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c4004800 msgr2=0x7f17c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.545+0000 7f17e2200640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c4004800 0x7f17c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.545+0000 7f17e1800640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40039b0 0x7f17c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.545+0000 7f17e2200640 1 -- 172.21.15.49:0/2932837582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40039b0 msgr2=0x7f17c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.545+0000 7f17e2200640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40039b0 0x7f17c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.545+0000 7f17e2200640 1 -- 172.21.15.49:0/2932837582 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c4003520 con 0x7f17c40051c0 2024-04-15T15:20:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.545+0000 7f17e0e00640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c4004800 0x7f17c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.545+0000 7f17e1800640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40039b0 0x7f17c4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.546+0000 7f17e2200640 1 -- 172.21.15.49:0/2932837582 reap_dead start 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.546+0000 7f17e2200640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c40051c0 0x7f17c400b9c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009fa0 tx=0x7f17c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44b16970a47995e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.547+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4039070 con 0x7f17c40051c0 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.547+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d402cce0 con 0x7f17c40051c0 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.547+0000 7f17dbe00640 1 -- 172.21.15.49:0/2932837582 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4009bf0 con 0x7f17c40051c0 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.547+0000 7f17e4200640 1 -- 172.21.15.49:0/2932837582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c40051c0 msgr2=0x7f17c400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.547+0000 7f17e4200640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c40051c0 0x7f17c400b9c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009fa0 tx=0x7f17c4005830 comp rx=0 tx=0).stop 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.547+0000 7f17e4200640 1 -- 172.21.15.49:0/2932837582 shutdown_connections 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.547+0000 7f17e4200640 1 --2- 172.21.15.49:0/2932837582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c40051c0 0x7f17c400b9c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.547+0000 7f17e4200640 1 -- 172.21.15.49:0/2932837582 >> 172.21.15.49:0/2932837582 conn(0x7f17dc0b27b0 msgr2=0x7f17dc0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.548+0000 7f17e4200640 1 -- 172.21.15.49:0/2932837582 shutdown_connections 2024-04-15T15:20:20.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.548+0000 7f17e4200640 1 -- 172.21.15.49:0/2932837582 wait complete. 2024-04-15T15:20:20.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.549+0000 7f17e4200640 1 Processor -- start 2024-04-15T15:20:20.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.549+0000 7f17e4200640 1 -- start start 2024-04-15T15:20:20.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.550+0000 7f17e4200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40051c0 0x7f17dc0b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.550+0000 7f17e4200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17dc14bc00 0x7f17dc1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.550+0000 7f17e4200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17dc155b10 0x7f17dc0ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e4200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc0cdd10 con 0x7f17dc155b10 2024-04-15T15:20:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e4200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc06f420 con 0x7f17c40051c0 2024-04-15T15:20:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e4200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f17dc0cde90 con 0x7f17dc14bc00 2024-04-15T15:20:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e2200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17dc155b10 0x7f17dc0ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e2200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17dc155b10 0x7f17dc0ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33172/0 (socket says 172.21.15.49:33172) 2024-04-15T15:20:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e2200640 1 -- 172.21.15.49:0/1104531744 learned_addr learned my addr 172.21.15.49:0/1104531744 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e0e00640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17dc14bc00 0x7f17dc1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e1800640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40051c0 0x7f17dc0b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e2200640 1 -- 172.21.15.49:0/1104531744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17dc14bc00 msgr2=0x7f17dc1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e2200640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17dc14bc00 0x7f17dc1555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e2200640 1 -- 172.21.15.49:0/1104531744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40051c0 msgr2=0x7f17dc0b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e2200640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40051c0 0x7f17dc0b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e2200640 1 -- 172.21.15.49:0/1104531744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc0ad8b0 con 0x7f17dc155b10 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e0e00640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17dc14bc00 0x7f17dc1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.551+0000 7f17e1800640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40051c0 0x7f17dc0b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.552+0000 7f17e2200640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17dc155b10 0x7f17dc0ae630 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f17d400e200 tx=0x7f17d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.552+0000 7f17d9600640 1 -- 172.21.15.49:0/1104531744 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4039040 con 0x7f17dc155b10 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.552+0000 7f17d9600640 1 -- 172.21.15.49:0/1104531744 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d4045070 con 0x7f17dc155b10 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.552+0000 7f17d9600640 1 -- 172.21.15.49:0/1104531744 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4009bf0 con 0x7f17dc155b10 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.552+0000 7f17e4200640 1 -- 172.21.15.49:0/1104531744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc156090 con 0x7f17dc155b10 2024-04-15T15:20:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.552+0000 7f17e4200640 1 -- 172.21.15.49:0/1104531744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc0ade30 con 0x7f17dc155b10 2024-04-15T15:20:20.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.554+0000 7f17e4200640 1 -- 172.21.15.49:0/1104531744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17dc068c80 con 0x7f17dc155b10 2024-04-15T15:20:20.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.555+0000 7f17d9600640 1 -- 172.21.15.49:0/1104531744 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f17d4005d10 con 0x7f17dc155b10 2024-04-15T15:20:20.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.556+0000 7f17d9600640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17bc06c9b0 0x7f17bc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:20.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.556+0000 7f17d9600640 1 -- 172.21.15.49:0/1104531744 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f17d40ad8f0 con 0x7f17dc155b10 2024-04-15T15:20:20.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.556+0000 7f17e1800640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17bc06c9b0 0x7f17bc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:20.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.557+0000 7f17e1800640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17bc06c9b0 0x7f17bc06ee70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f17dc068410 tx=0x7f17d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.561+0000 7f17d9600640 1 -- 172.21.15.49:0/1104531744 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f17d407bc30 con 0x7f17dc155b10 2024-04-15T15:20:20.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.816+0000 7f17e4200640 1 -- 172.21.15.49:0/1104531744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f17dc0b0980 con 0x7f17dc155b10 2024-04-15T15:20:20.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.817+0000 7f17d9600640 1 -- 172.21.15.49:0/1104531744 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f17d407b530 con 0x7f17dc155b10 2024-04-15T15:20:20.819 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 -- 172.21.15.49:0/1104531744 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17bc06c9b0 msgr2=0x7f17bc06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17bc06c9b0 0x7f17bc06ee70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f17dc068410 tx=0x7f17d0009040 comp rx=0 tx=0).stop 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 -- 172.21.15.49:0/1104531744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17dc155b10 msgr2=0x7f17dc0ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17dc155b10 0x7f17dc0ae630 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f17d400e200 tx=0x7f17d4009750 comp rx=0 tx=0).stop 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 -- 172.21.15.49:0/1104531744 shutdown_connections 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f17bc06c9b0 0x7f17bc06ee70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17dc14bc00 0x7f17dc1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17dc155b10 0x7f17dc0ae630 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 --2- 172.21.15.49:0/1104531744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40051c0 0x7f17dc0b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 -- 172.21.15.49:0/1104531744 >> 172.21.15.49:0/1104531744 conn(0x7f17dc0b27b0 msgr2=0x7f17dc156920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 -- 172.21.15.49:0/1104531744 shutdown_connections 2024-04-15T15:20:20.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:20.821+0000 7f17e4200640 1 -- 172.21.15.49:0/1104531744 wait complete. 2024-04-15T15:20:20.842 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:20:20.842 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:20.842 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-04-15T15:20:20.842 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-15T15:20:20.842 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:20.843 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:20:20.843 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:20.843 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:20.843 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:20.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:20.843 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-04-15T15:20:20.902 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:20.902 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:20.902 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:20.902 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:20.902 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:20.902 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:20.902 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:20.902 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42501sec preferred_lft 42501sec 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 82:38:d9:8f:e0:b7 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:20.903 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::26:49ff:feaf:7d0d/64 scope link 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether fe:39:d9:9f:a9:b1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::fc39:d9ff:fe9f:a9b1/64 scope link 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 82:38:d9:8f:e0:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::8038:d9ff:fe8f:e0b7/64 scope link 2024-04-15T15:20:20.904 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:20.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:20.905 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:20.990 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-15T15:20:20.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:20.994 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-04-15T15:20:21.045 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-04-15T15:20:21.046 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:21.046 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:20:21.046 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:20:21.046 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:21.046 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:21.046 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:20:21.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:20:21.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:20:21.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:20:21.046 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:20:21.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:21.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:21.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:21.286 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-15T15:20:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:20:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:20:21.336 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:21.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:21.337 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-04-15T15:20:21.422 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:21.422 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:21.474 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:20:21.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:21.474 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:21.524 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:21.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:21.524 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:20:21.659 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:20:21.660 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:20:21.660 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:20:21.682 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:20:21.682 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:20:21.682 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:20:21.682 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:21.682 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:21.711 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:20:21.711 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:20:21.711 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:20:21.712 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:21.712 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:21.712 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790" 2024-04-15T15:20:21.712 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790:/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:21.712 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c" will pass to kernel 2024-04-15T15:20:21.713 INFO:teuthology.orchestra.run.smithi107.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-15T15:20:21.713 INFO:teuthology.orchestra.run.smithi107.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:20:21.713 INFO:teuthology.orchestra.run.smithi107.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:20:21.713 INFO:teuthology.orchestra.run.smithi107.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:20:21.713 INFO:teuthology.orchestra.run.smithi107.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:20:21.720 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:20:21.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:21.720 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:21.798 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:20:21.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:20:21.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:21.912 INFO:teuthology.orchestra.run.smithi107.stdout:client5370 2024-04-15T15:20:21.913 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:22.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.147+0000 7fd57f200640 1 Processor -- start 2024-04-15T15:20:22.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.147+0000 7fd57f200640 1 -- start start 2024-04-15T15:20:22.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.149+0000 7fd57f200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd5800cee00 0x7fd58006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:22.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.149+0000 7fd57f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58015d2f0 0x7fd5800723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:22.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.149+0000 7fd57f200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd58015dbb0 con 0x7fd580071f50 2024-04-15T15:20:22.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.149+0000 7fd57f200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fd58006f4f0 con 0x7fd5800cee00 2024-04-15T15:20:22.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.149+0000 7fd57f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd5800728f0 con 0x7fd58015d2f0 2024-04-15T15:20:22.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.150+0000 7fd57de00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd5800cee00 0x7fd58006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:22.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.150+0000 7fd57d400640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fd580071f50 0x7fd58015cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:33564/0 (socket says 172.21.15.49:33564) 2024-04-15T15:20:22.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.150+0000 7fd57de00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd5800cee00 0x7fd58006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:55448/0 (socket says 172.21.15.49:55448) 2024-04-15T15:20:22.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.150+0000 7fd57d400640 1 -- 172.21.15.49:0/1866059562 learned_addr learned my addr 172.21.15.49:0/1866059562 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:22.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.150+0000 7fd57e800640 1 --2- 172.21.15.49:0/1866059562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58015d2f0 0x7fd5800723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:22.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.151+0000 7fd57de00640 1 -- 172.21.15.49:0/1866059562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58015d2f0 msgr2=0x7fd5800723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:22.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.151+0000 7fd57de00640 1 --2- 172.21.15.49:0/1866059562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58015d2f0 0x7fd5800723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.151+0000 7fd57de00640 1 -- 172.21.15.49:0/1866059562 >> v1:172.21.15.107:6789/0 conn(0x7fd580071f50 legacy=0x7fd58015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:22.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.151+0000 7fd57de00640 1 -- 172.21.15.49:0/1866059562 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58015d8d0 con 0x7fd5800cee00 2024-04-15T15:20:22.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.152+0000 7fd57de00640 1 --2- 172.21.15.49:0/1866059562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5800cee00 0x7fd58006e8c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5700097a0 tx=0x7fd57002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e6727ea533f4c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:22.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.152+0000 7fd57ca00640 1 -- 172.21.15.49:0/1866059562 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570035020 con 0x7fd5800cee00 2024-04-15T15:20:22.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.153+0000 7fd57ca00640 1 -- 172.21.15.49:0/1866059562 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd570031810 con 0x7fd5800cee00 2024-04-15T15:20:22.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.153+0000 7fd57ca00640 1 -- 172.21.15.49:0/1866059562 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570031be0 con 0x7fd5800cee00 2024-04-15T15:20:22.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.153+0000 7fd57f200640 1 -- 172.21.15.49:0/1866059562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5800cee00 msgr2=0x7fd58006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:22.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.153+0000 7fd57f200640 1 --2- 172.21.15.49:0/1866059562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5800cee00 0x7fd58006e8c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5700097a0 tx=0x7fd57002b8b0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.153+0000 7fd57f200640 1 -- 172.21.15.49:0/1866059562 shutdown_connections 2024-04-15T15:20:22.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.153+0000 7fd57f200640 1 --2- 172.21.15.49:0/1866059562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58015d2f0 0x7fd5800723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.153+0000 7fd57f200640 1 --2- 172.21.15.49:0/1866059562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5800cee00 0x7fd58006e8c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.154+0000 7fd57f200640 1 -- 172.21.15.49:0/1866059562 >> 172.21.15.49:0/1866059562 conn(0x7fd58006b930 msgr2=0x7fd58006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:22.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.154+0000 7fd57f200640 1 -- 172.21.15.49:0/1866059562 shutdown_connections 2024-04-15T15:20:22.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.154+0000 7fd57f200640 1 -- 172.21.15.49:0/1866059562 wait complete. 2024-04-15T15:20:22.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.156+0000 7fd57f200640 1 Processor -- start 2024-04-15T15:20:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.156+0000 7fd57f200640 1 -- start start 2024-04-15T15:20:22.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.157+0000 7fd57f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd580071f50 0x7fd58013efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:22.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.158+0000 7fd57f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5800cee00 0x7fd58014e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:22.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.158+0000 7fd57f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58015d2f0 0x7fd58014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:22.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.158+0000 7fd57f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd58006e3a0 con 0x7fd58015d2f0 2024-04-15T15:20:22.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.158+0000 7fd57f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd58006e0c0 con 0x7fd580071f50 2024-04-15T15:20:22.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.158+0000 7fd57f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd58015dda0 con 0x7fd5800cee00 2024-04-15T15:20:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.158+0000 7fd57e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58015d2f0 0x7fd58014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.158+0000 7fd57e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58015d2f0 0x7fd58014eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33176/0 (socket says 172.21.15.49:33176) 2024-04-15T15:20:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.158+0000 7fd57e800640 1 -- 172.21.15.49:0/99859393 learned_addr learned my addr 172.21.15.49:0/99859393 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.158+0000 7fd57de00640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd580071f50 0x7fd58013efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.158+0000 7fd57d400640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5800cee00 0x7fd58014e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd57e800640 1 -- 172.21.15.49:0/99859393 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5800cee00 msgr2=0x7fd58014e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd57e800640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5800cee00 0x7fd58014e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd57e800640 1 -- 172.21.15.49:0/99859393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd580071f50 msgr2=0x7fd58013efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd57e800640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd580071f50 0x7fd58013efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd57e800640 1 -- 172.21.15.49:0/99859393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58014d4e0 con 0x7fd58015d2f0 2024-04-15T15:20:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd57d400640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5800cee00 0x7fd58014e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd57e800640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58015d2f0 0x7fd58014eef0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd578009d30 tx=0x7fd57800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd576000640 1 -- 172.21.15.49:0/99859393 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578018020 con 0x7fd58015d2f0 2024-04-15T15:20:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd576000640 1 -- 172.21.15.49:0/99859393 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd578014480 con 0x7fd58015d2f0 2024-04-15T15:20:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd576000640 1 -- 172.21.15.49:0/99859393 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578014870 con 0x7fd58015d2f0 2024-04-15T15:20:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.159+0000 7fd57f200640 1 -- 172.21.15.49:0/99859393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58014d7c0 con 0x7fd58015d2f0 2024-04-15T15:20:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.160+0000 7fd57f200640 1 -- 172.21.15.49:0/99859393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58014dd80 con 0x7fd58015d2f0 2024-04-15T15:20:22.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.161+0000 7fd57f200640 1 -- 172.21.15.49:0/99859393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5800755c0 con 0x7fd58015d2f0 2024-04-15T15:20:22.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.162+0000 7fd576000640 1 -- 172.21.15.49:0/99859393 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd578005d10 con 0x7fd58015d2f0 2024-04-15T15:20:22.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.163+0000 7fd576000640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd56006c980 0x7fd56006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:22.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.163+0000 7fd576000640 1 -- 172.21.15.49:0/99859393 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd57808cb70 con 0x7fd58015d2f0 2024-04-15T15:20:22.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.163+0000 7fd57de00640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd56006c980 0x7fd56006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:22.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.164+0000 7fd57de00640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd56006c980 0x7fd56006ee40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd570036000 tx=0x7fd570008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:22.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.167+0000 7fd576000640 1 -- 172.21.15.49:0/99859393 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd578022030 con 0x7fd58015d2f0 2024-04-15T15:20:22.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.420+0000 7fd57f200640 1 -- 172.21.15.49:0/99859393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd58015ef20 con 0x7fd58015d2f0 2024-04-15T15:20:22.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.421+0000 7fd576000640 1 -- 172.21.15.49:0/99859393 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fd578010040 con 0x7fd58015d2f0 2024-04-15T15:20:22.423 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 -- 172.21.15.49:0/99859393 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd56006c980 msgr2=0x7fd56006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd56006c980 0x7fd56006ee40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd570036000 tx=0x7fd570008030 comp rx=0 tx=0).stop 2024-04-15T15:20:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 -- 172.21.15.49:0/99859393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58015d2f0 msgr2=0x7fd58014eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58015d2f0 0x7fd58014eef0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd578009d30 tx=0x7fd57800c490 comp rx=0 tx=0).stop 2024-04-15T15:20:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 -- 172.21.15.49:0/99859393 shutdown_connections 2024-04-15T15:20:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd56006c980 0x7fd56006ee40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5800cee00 0x7fd58014e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58015d2f0 0x7fd58014eef0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 --2- 172.21.15.49:0/99859393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd580071f50 0x7fd58013efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:22.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 -- 172.21.15.49:0/99859393 >> 172.21.15.49:0/99859393 conn(0x7fd58006b930 msgr2=0x7fd580074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:22.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.424+0000 7fd57f200640 1 -- 172.21.15.49:0/99859393 shutdown_connections 2024-04-15T15:20:22.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22.425+0000 7fd57f200640 1 -- 172.21.15.49:0/99859393 wait complete. 2024-04-15T15:20:22.446 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:20:22.446 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:22.446 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-04-15T15:20:22.447 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-15T15:20:22.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-15T15:20:22.447 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:20:22.447 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:22.447 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:22.447 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:20:22.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:22.447 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-04-15T15:20:22.502 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:22.502 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:22.502 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:22.502 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:22.502 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:22.502 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:22.502 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:22.503 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:22.503 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-04-15T15:20:22.503 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:22.503 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:22.503 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-04-15T15:20:22.505 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:22.505 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:22.505 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42480sec preferred_lft 42480sec 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 3a:d8:a3:bb:09:2d brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:22.506 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:22.507 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::6c47:44ff:feb2:eb9c/64 scope link 2024-04-15T15:20:22.507 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:22.507 INFO:teuthology.orchestra.run.smithi049.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:22.507 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 3a:d8:a3:bb:09:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:22.507 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::38d8:a3ff:febb:92d/64 scope link 2024-04-15T15:20:22.507 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:22.507 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:22.507 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 66:4c:0b:61:1e:4f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:22.507 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::644c:bff:fe61:1e4f/64 scope link 2024-04-15T15:20:22.507 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:22.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:22.509 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:22.612 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-04-15T15:20:22.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:22.617 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-04-15T15:20:22.676 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-04-15T15:20:22.677 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:22.677 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-15T15:20:22.677 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-15T15:20:22.677 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:22.677 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:22.677 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-04-15T15:20:22.677 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-15T15:20:22.677 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-15T15:20:22.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-15T15:20:22.678 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-04-15T15:20:22.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:22.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:22.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:22.952 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-04-15T15:20:23.006 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:20:23.006 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:20:23.007 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:20:23.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:23.007 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.1 up 2024-04-15T15:20:23.113 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:23.113 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-15T15:20:23.174 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:20:23.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:23.175 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:20:23.233 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:23.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:23.233 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:20:23.386 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:20:23.386 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:20:23.386 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:20:23.409 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:20:23.409 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:20:23.409 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:20:23.409 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:23.409 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:23.435 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:20:23.435 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:20:23.435 INFO:teuthology.orchestra.run.smithi049.stdout:invalid new device string format 2024-04-15T15:20:23.435 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:23.435 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:23.435 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790" 2024-04-15T15:20:23.436 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790:/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:23.436 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c" will pass to kernel 2024-04-15T15:20:23.438 INFO:teuthology.orchestra.run.smithi049.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-15T15:20:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:20:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:20:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:20:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:20:23.443 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:20:23.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:23.443 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:20:23.535 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:20:23.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:20:23.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:23.661 INFO:teuthology.orchestra.run.smithi049.stdout:client5017 2024-04-15T15:20:23.661 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:20:23.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.889+0000 7f4e58c00640 1 Processor -- start 2024-04-15T15:20:23.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.889+0000 7f4e58c00640 1 -- start start 2024-04-15T15:20:23.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.890+0000 7f4e58c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4e54162020 0x7f4e54162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.890+0000 7f4e58c00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e540d0090 con 0x7f4e540d1c90 2024-04-15T15:20:23.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.890+0000 7f4e58c00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e540cfad0 con 0x7f4e540cf110 2024-04-15T15:20:23.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.890+0000 7f4e58c00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f4e54163440 con 0x7f4e54162020 2024-04-15T15:20:23.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.890+0000 7f4e53400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4e54162020 0x7f4e54162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.890+0000 7f4e53400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4e54162020 0x7f4e54162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33184/0 (socket says 172.21.15.49:33184) 2024-04-15T15:20:23.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.890+0000 7f4e53400640 1 -- 172.21.15.49:0/4066012853 learned_addr learned my addr 172.21.15.49:0/4066012853 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:23.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.890+0000 7f4e53e00640 1 --1- 172.21.15.49:0/4066012853 >> v1:172.21.15.49:6789/0 conn(0x7f4e540d1c90 0x7f4e540d3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55134/0 (socket says 172.21.15.49:55134) 2024-04-15T15:20:23.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.891+0000 7f4e53400640 1 -- 172.21.15.49:0/4066012853 >> v1:172.21.15.107:6789/0 conn(0x7f4e540cf110 legacy=0x7f4e540d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:23.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.891+0000 7f4e53400640 1 -- 172.21.15.49:0/4066012853 >> v1:172.21.15.49:6789/0 conn(0x7f4e540d1c90 legacy=0x7f4e540d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.891+0000 7f4e53400640 1 -- 172.21.15.49:0/4066012853 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e540cfdb0 con 0x7f4e54162020 2024-04-15T15:20:23.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.892+0000 7f4e53400640 1 --2- 172.21.15.49:0/4066012853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 0x7f4e54162f00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009870 tx=0x7f4e3c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9205bd3125b55ab server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.892+0000 7f4e52000640 1 -- 172.21.15.49:0/4066012853 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c034070 con 0x7f4e54162020 2024-04-15T15:20:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.892+0000 7f4e52000640 1 -- 172.21.15.49:0/4066012853 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3c030860 con 0x7f4e54162020 2024-04-15T15:20:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.892+0000 7f4e52000640 1 -- 172.21.15.49:0/4066012853 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c030c30 con 0x7f4e54162020 2024-04-15T15:20:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.893+0000 7f4e58c00640 1 -- 172.21.15.49:0/4066012853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 msgr2=0x7f4e54162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.893+0000 7f4e58c00640 1 --2- 172.21.15.49:0/4066012853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 0x7f4e54162f00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009870 tx=0x7f4e3c02b9a0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.893+0000 7f4e58c00640 1 -- 172.21.15.49:0/4066012853 shutdown_connections 2024-04-15T15:20:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.893+0000 7f4e58c00640 1 --2- 172.21.15.49:0/4066012853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 0x7f4e54162f00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.893+0000 7f4e58c00640 1 -- 172.21.15.49:0/4066012853 >> 172.21.15.49:0/4066012853 conn(0x7f4e5406cf00 msgr2=0x7f4e5406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:23.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.893+0000 7f4e58c00640 1 -- 172.21.15.49:0/4066012853 shutdown_connections 2024-04-15T15:20:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.893+0000 7f4e58c00640 1 -- 172.21.15.49:0/4066012853 wait complete. 2024-04-15T15:20:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.895+0000 7f4e58c00640 1 Processor -- start 2024-04-15T15:20:23.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.895+0000 7f4e58c00640 1 -- start start 2024-04-15T15:20:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.896+0000 7f4e58c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e540cf110 0x7f4e5415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.896+0000 7f4e58c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e540d1c90 0x7f4e54157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.896+0000 7f4e58c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 0x7f4e540cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.896+0000 7f4e58c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540cf880 con 0x7f4e54162020 2024-04-15T15:20:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.896+0000 7f4e58c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540b7b30 con 0x7f4e540d1c90 2024-04-15T15:20:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.896+0000 7f4e58c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f4e540d2e70 con 0x7f4e540cf110 2024-04-15T15:20:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.896+0000 7f4e53e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 0x7f4e540cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.896+0000 7f4e53e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 0x7f4e540cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33190/0 (socket says 172.21.15.49:33190) 2024-04-15T15:20:23.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.896+0000 7f4e53e00640 1 -- 172.21.15.49:0/1183207698 learned_addr learned my addr 172.21.15.49:0/1183207698 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e52a00640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e540d1c90 0x7f4e54157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e53400640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e540cf110 0x7f4e5415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e53e00640 1 -- 172.21.15.49:0/1183207698 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e540cf110 msgr2=0x7f4e5415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e53e00640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e540cf110 0x7f4e5415ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e53e00640 1 -- 172.21.15.49:0/1183207698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e540d1c90 msgr2=0x7f4e54157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e53e00640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e540d1c90 0x7f4e54157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e53e00640 1 -- 172.21.15.49:0/1183207698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e54161230 con 0x7f4e54162020 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e53400640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e540cf110 0x7f4e5415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e53e00640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 0x7f4e540cdd80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00c6c0 tx=0x7f4e4c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e4b400640 1 -- 172.21.15.49:0/1183207698 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c018070 con 0x7f4e54162020 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e4b400640 1 -- 172.21.15.49:0/1183207698 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e4c004d60 con 0x7f4e54162020 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.897+0000 7f4e4b400640 1 -- 172.21.15.49:0/1183207698 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00a840 con 0x7f4e54162020 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.898+0000 7f4e58c00640 1 -- 172.21.15.49:0/1183207698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e54159dc0 con 0x7f4e54162020 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.898+0000 7f4e58c00640 1 -- 172.21.15.49:0/1183207698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5415a380 con 0x7f4e54162020 2024-04-15T15:20:23.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.899+0000 7f4e58c00640 1 -- 172.21.15.49:0/1183207698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e540755c0 con 0x7f4e54162020 2024-04-15T15:20:23.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.900+0000 7f4e4b400640 1 -- 172.21.15.49:0/1183207698 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4e4c00a9e0 con 0x7f4e54162020 2024-04-15T15:20:23.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.901+0000 7f4e4b400640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4e3006c980 0x7f4e3006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:23.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.902+0000 7f4e4b400640 1 -- 172.21.15.49:0/1183207698 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f4e4c08b090 con 0x7f4e54162020 2024-04-15T15:20:23.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.902+0000 7f4e53400640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4e3006c980 0x7f4e3006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:23.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.903+0000 7f4e53400640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4e3006c980 0x7f4e3006ee40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c036000 tx=0x7f4e3c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:23.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:23.906+0000 7f4e4b400640 1 -- 172.21.15.49:0/1183207698 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e540d0090 con 0x7f4e54162020 2024-04-15T15:20:24.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.150+0000 7f4e58c00640 1 -- 172.21.15.49:0/1183207698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4e5415e0e0 con 0x7f4e54162020 2024-04-15T15:20:24.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.151+0000 7f4e4b400640 1 -- 172.21.15.49:0/1183207698 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f4e4c058d80 con 0x7f4e54162020 2024-04-15T15:20:24.153 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:20:24.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 -- 172.21.15.49:0/1183207698 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4e3006c980 msgr2=0x7f4e3006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4e3006c980 0x7f4e3006ee40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c036000 tx=0x7f4e3c03b040 comp rx=0 tx=0).stop 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 -- 172.21.15.49:0/1183207698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 msgr2=0x7f4e540cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 0x7f4e540cdd80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00c6c0 tx=0x7f4e4c00cb80 comp rx=0 tx=0).stop 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 -- 172.21.15.49:0/1183207698 shutdown_connections 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f4e3006c980 0x7f4e3006ee40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e54162020 0x7f4e540cdd80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e540d1c90 0x7f4e54157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 --2- 172.21.15.49:0/1183207698 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e540cf110 0x7f4e5415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 -- 172.21.15.49:0/1183207698 >> 172.21.15.49:0/1183207698 conn(0x7f4e5406cf00 msgr2=0x7f4e540ca360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 -- 172.21.15.49:0/1183207698 shutdown_connections 2024-04-15T15:20:24.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:24.154+0000 7f4e58c00640 1 -- 172.21.15.49:0/1183207698 wait complete. 2024-04-15T15:20:24.175 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:20:24.175 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:20:24.175 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-04-15T15:20:24.175 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-15T15:20:24.176 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-15T15:20:24.176 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:20:24.176 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:24.176 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:20:24.176 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:20:24.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:24.176 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-04-15T15:20:24.234 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-04-15T15:20:24.235 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42498sec preferred_lft 42498sec 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 82:38:d9:8f:e0:b7 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:20:24.236 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:24.237 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::26:49ff:feaf:7d0d/64 scope link 2024-04-15T15:20:24.237 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:24.237 INFO:teuthology.orchestra.run.smithi107.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:24.237 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether fe:39:d9:9f:a9:b1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:24.237 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::fc39:d9ff:fe9f:a9b1/64 scope link 2024-04-15T15:20:24.237 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:24.237 INFO:teuthology.orchestra.run.smithi107.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:20:24.237 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 82:38:d9:8f:e0:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:24.237 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::8038:d9ff:fe8f:e0b7/64 scope link 2024-04-15T15:20:24.237 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:20:24.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:24.239 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:20:24.330 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-15T15:20:24.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:24.334 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-04-15T15:20:24.383 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-04-15T15:20:24.384 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:20:24.384 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:20:24.384 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:20:24.384 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:20:24.385 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:20:24.385 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-15T15:20:24.385 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:20:24.385 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:20:24.385 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:20:24.385 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-15T15:20:24.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:20:24.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:24.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:24.630 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-15T15:20:24.676 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:20:24.676 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:20:24.677 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:20:24.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:24.677 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.1 up 2024-04-15T15:20:24.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:24.814 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-15T15:20:24.870 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:20:24.870 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:20:24.870 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:20:24.919 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:24.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:24.919 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:20:25.051 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:20:25.051 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:20:25.051 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:20:25.074 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:20:25.074 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:20:25.074 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:20:25.074 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:25.074 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:25.103 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:20:25.103 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:20:25.104 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-15T15:20:25.104 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2024-04-15T15:20:25.104 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:20:25.104 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790" 2024-04-15T15:20:25.104 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.107:6789,172.21.15.107:6790:/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 2024-04-15T15:20:25.104 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c" will pass to kernel 2024-04-15T15:20:25.106 INFO:teuthology.orchestra.run.smithi107.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-15T15:20:25.106 INFO:teuthology.orchestra.run.smithi107.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:20:25.106 INFO:teuthology.orchestra.run.smithi107.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:20:25.106 INFO:teuthology.orchestra.run.smithi107.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:20:25.106 INFO:teuthology.orchestra.run.smithi107.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:20:25.110 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:20:25.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:25.110 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:20:25.191 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:20:25.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:20:25.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-15T15:20:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:25.313 INFO:teuthology.orchestra.run.smithi107.stdout:client5023 2024-04-15T15:20:25.313 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'touch file') 2024-04-15T15:20:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:20:25.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:20:25.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:25.472 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quiesce.block -v 1 file) 2024-04-15T15:20:25.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 quiesce path /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b --wait 2024-04-15T15:20:25.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.791+0000 7f68e7e00640 1 Processor -- start 2024-04-15T15:20:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.792+0000 7f68e7e00640 1 -- start start 2024-04-15T15:20:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.793+0000 7f68e7e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68e80cf510 0x7f68e80b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.793+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e80b78e0 0x7f68e80b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.793+0000 7f68e7e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e80d3350 con 0x7f68e80d2b90 2024-04-15T15:20:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.793+0000 7f68e7e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f68e80d3f60 con 0x7f68e80cf510 2024-04-15T15:20:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.793+0000 7f68e7e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f68e80d3630 con 0x7f68e80b78e0 2024-04-15T15:20:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.794+0000 7f68e6a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68e80cf510 0x7f68e80b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.794+0000 7f68e6a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68e80cf510 0x7f68e80b6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43516/0 (socket says 172.21.15.49:43516) 2024-04-15T15:20:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.794+0000 7f68e6a00640 1 -- 172.21.15.49:0/2029852658 learned_addr learned my addr 172.21.15.49:0/2029852658 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.794+0000 7f68e6a00640 1 -- 172.21.15.49:0/2029852658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e80b78e0 msgr2=0x7f68e80b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.794+0000 7f68e6a00640 1 --2- 172.21.15.49:0/2029852658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e80b78e0 0x7f68e80b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.794+0000 7f68e6a00640 1 -- 172.21.15.49:0/2029852658 >> v1:172.21.15.107:6789/0 conn(0x7f68e80d2b90 legacy=0x7f68e80cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.794+0000 7f68e6a00640 1 -- 172.21.15.49:0/2029852658 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e80d3910 con 0x7f68e80cf510 2024-04-15T15:20:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.795+0000 7f68e6a00640 1 --2- 172.21.15.49:0/2029852658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80cf510 0x7f68e80b6b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28376fbcfe34b02f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.796+0000 7f68e5600640 1 -- 172.21.15.49:0/2029852658 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8035020 con 0x7f68e80cf510 2024-04-15T15:20:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.796+0000 7f68e5600640 1 -- 172.21.15.49:0/2029852658 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d8031640 con 0x7f68e80cf510 2024-04-15T15:20:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.796+0000 7f68e5600640 1 -- 172.21.15.49:0/2029852658 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d802d040 con 0x7f68e80cf510 2024-04-15T15:20:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.797+0000 7f68e7e00640 1 -- 172.21.15.49:0/2029852658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80cf510 msgr2=0x7f68e80b6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.797+0000 7f68e7e00640 1 --2- 172.21.15.49:0/2029852658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80cf510 0x7f68e80b6b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.797+0000 7f68e7e00640 1 -- 172.21.15.49:0/2029852658 shutdown_connections 2024-04-15T15:20:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.797+0000 7f68e7e00640 1 --2- 172.21.15.49:0/2029852658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e80b78e0 0x7f68e80b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.797+0000 7f68e7e00640 1 --2- 172.21.15.49:0/2029852658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80cf510 0x7f68e80b6b80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.797+0000 7f68e7e00640 1 -- 172.21.15.49:0/2029852658 >> 172.21.15.49:0/2029852658 conn(0x7f68e80d13a0 msgr2=0x7f68e80d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.797+0000 7f68e7e00640 1 -- 172.21.15.49:0/2029852658 shutdown_connections 2024-04-15T15:20:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.797+0000 7f68e7e00640 1 -- 172.21.15.49:0/2029852658 wait complete. 2024-04-15T15:20:25.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.799+0000 7f68e7e00640 1 Processor -- start 2024-04-15T15:20:25.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.799+0000 7f68e7e00640 1 -- start start 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e80b78e0 0x7f68e80b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e80cf510 0x7f68e81e7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80d2b90 0x7f68e81e7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e7e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68e80d3130 con 0x7f68e80d2b90 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e7e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f68e80d2fb0 con 0x7f68e80cf510 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e7e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f68e80d43f0 con 0x7f68e80b78e0 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80d2b90 0x7f68e81e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80d2b90 0x7f68e81e7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43528/0 (socket says 172.21.15.49:43528) 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e7400640 1 -- 172.21.15.49:0/2888099332 learned_addr learned my addr 172.21.15.49:0/2888099332 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e6000640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e80cf510 0x7f68e81e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.801+0000 7f68e6a00640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e80b78e0 0x7f68e80b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.802+0000 7f68e7400640 1 -- 172.21.15.49:0/2888099332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e80b78e0 msgr2=0x7f68e80b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.802+0000 7f68e7400640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e80b78e0 0x7f68e80b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.802+0000 7f68e7400640 1 -- 172.21.15.49:0/2888099332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e80cf510 msgr2=0x7f68e81e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.802+0000 7f68e7400640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e80cf510 0x7f68e81e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.802+0000 7f68e7400640 1 -- 172.21.15.49:0/2888099332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e81e49d0 con 0x7f68e80d2b90 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.802+0000 7f68e6000640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e80cf510 0x7f68e81e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.802+0000 7f68e7400640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80d2b90 0x7f68e81e7880 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f68e000b410 tx=0x7f68e000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.802+0000 7f68dea00640 1 -- 172.21.15.49:0/2888099332 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0015020 con 0x7f68e80d2b90 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.803+0000 7f68dea00640 1 -- 172.21.15.49:0/2888099332 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68e0011850 con 0x7f68e80d2b90 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.803+0000 7f68e7e00640 1 -- 172.21.15.49:0/2888099332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e80ce180 con 0x7f68e80d2b90 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.803+0000 7f68dea00640 1 -- 172.21.15.49:0/2888099332 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0011c40 con 0x7f68e80d2b90 2024-04-15T15:20:25.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.803+0000 7f68e7e00640 1 -- 172.21.15.49:0/2888099332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e814b840 con 0x7f68e80d2b90 2024-04-15T15:20:25.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.805+0000 7f68dea00640 1 -- 172.21.15.49:0/2888099332 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f68e0018020 con 0x7f68e80d2b90 2024-04-15T15:20:25.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.806+0000 7f68dea00640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f68c406c9b0 0x7f68c406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.807+0000 7f68dea00640 1 -- 172.21.15.49:0/2888099332 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f68e008bbb0 con 0x7f68e80d2b90 2024-04-15T15:20:25.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.807+0000 7f68e6a00640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f68c406c9b0 0x7f68c406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.807+0000 7f68f03cd740 1 Processor -- start 2024-04-15T15:20:25.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.807+0000 7f68f03cd740 1 -- start start 2024-04-15T15:20:25.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.808+0000 7f68e6a00640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f68c406c9b0 0x7f68c406ee70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f68d8036000 tx=0x7f68d803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.808+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x564677048800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.808+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x5646770785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.808+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677048d40 0x564677078b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.808+0000 7f68f03cd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564676fb3540 con 0x564677075860 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.808+0000 7f68f03cd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564676f59b70 con 0x564677048d40 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.808+0000 7f68f03cd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x564676faf500 con 0x564677075c40 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e6a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x564677048800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e6a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x564677048800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43538/0 (socket says 172.21.15.49:43538) 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e6a00640 1 -- 172.21.15.49:0/684292531 learned_addr learned my addr 172.21.15.49:0/684292531 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e7400640 1 --2- 172.21.15.49:0/684292531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677048d40 0x564677078b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e6a00640 1 -- 172.21.15.49:0/684292531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 msgr2=0x5646770785f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e6a00640 1 --2- 172.21.15.49:0/684292531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x5646770785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e6a00640 1 -- 172.21.15.49:0/684292531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677048d40 msgr2=0x564677078b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e6a00640 1 --2- 172.21.15.49:0/684292531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677048d40 0x564677078b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e6a00640 1 -- 172.21.15.49:0/684292531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564676f3ede0 con 0x564677075860 2024-04-15T15:20:25.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e7400640 1 --2- 172.21.15.49:0/684292531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677048d40 0x564677078b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:25.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.809+0000 7f68e6a00640 1 --2- 172.21.15.49:0/684292531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x564677048800 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f68d80097c0 tx=0x7f68d8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e31c1e5f09c882c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.810+0000 7f68d7000640 1 -- 172.21.15.49:0/684292531 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8053070 con 0x564677075860 2024-04-15T15:20:25.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.810+0000 7f68d7000640 1 -- 172.21.15.49:0/684292531 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d803a5a0 con 0x564677075860 2024-04-15T15:20:25.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.810+0000 7f68d7000640 1 -- 172.21.15.49:0/684292531 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d803a950 con 0x564677075860 2024-04-15T15:20:25.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.810+0000 7f68f03cd740 1 -- 172.21.15.49:0/684292531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 msgr2=0x564677048800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.810+0000 7f68f03cd740 1 --2- 172.21.15.49:0/684292531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x564677048800 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f68d80097c0 tx=0x7f68d8008030 comp rx=0 tx=0).stop 2024-04-15T15:20:25.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.811+0000 7f68f03cd740 1 -- 172.21.15.49:0/684292531 shutdown_connections 2024-04-15T15:20:25.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.811+0000 7f68f03cd740 1 --2- 172.21.15.49:0/684292531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x5646770785f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.811+0000 7f68f03cd740 1 --2- 172.21.15.49:0/684292531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677048d40 0x564677078b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.811+0000 7f68f03cd740 1 --2- 172.21.15.49:0/684292531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x564677048800 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.811+0000 7f68f03cd740 1 -- 172.21.15.49:0/684292531 >> 172.21.15.49:0/684292531 conn(0x56467704aa70 msgr2=0x5646770935f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.811+0000 7f68f03cd740 1 -- 172.21.15.49:0/684292531 shutdown_connections 2024-04-15T15:20:25.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.811+0000 7f68f03cd740 1 -- 172.21.15.49:0/684292531 wait complete. 2024-04-15T15:20:25.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.811+0000 7f68f03cd740 1 -- start start 2024-04-15T15:20:25.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.812+0000 7f68f03cd740 1 Processor -- start 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x56467706e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x5646770a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646770a4100 0x564677077cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68f03cd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56467704d010 con 0x564677075860 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68f03cd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564676f6cf40 con 0x5646770a4100 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68f03cd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x56467704c400 con 0x564677075c40 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68f03cd740 10 client.0 _ll_register_callbacks cb 0x564676f56550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68e6a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x56467706e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68e6a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x56467706e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43552/0 (socket says 172.21.15.49:43552) 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68e6a00640 1 -- 172.21.15.49:0/4002715922 learned_addr learned my addr 172.21.15.49:0/4002715922 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:25.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68e6a00640 1 -- 172.21.15.49:0/4002715922 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 msgr2=0x5646770a3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68e6a00640 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x5646770a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68e6a00640 1 -- 172.21.15.49:0/4002715922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646770a4100 msgr2=0x564677077cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68e6a00640 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646770a4100 0x564677077cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.813+0000 7f68e6a00640 1 -- 172.21.15.49:0/4002715922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564676d92030 con 0x564677075860 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.814+0000 7f68e6a00640 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x56467706e740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009080 tx=0x7f68d802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b141f157fb949bb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.814+0000 7f68cf400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.814+0000 7f68cf400640 1 -- 172.21.15.49:0/4002715922 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8053040 con 0x564677075860 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.814+0000 7f68cf400640 1 -- 172.21.15.49:0/4002715922 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f68d8059420 con 0x564677075860 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.814+0000 7f68f03cd740 1 -- 172.21.15.49:0/4002715922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d803a5a0 con 0x564677075860 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.814+0000 7f68cf400640 1 -- 172.21.15.49:0/4002715922 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f68d8059620 con 0x564677075860 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.814+0000 7f68f03cd740 10 client.5394 fetch_fsmap learned FSMap version 17 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.814+0000 7f68f03cd740 1 -- 172.21.15.49:0/4002715922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x564676cc5c20 con 0x564677075860 2024-04-15T15:20:25.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.815+0000 7f68cf400640 1 -- 172.21.15.49:0/4002715922 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d803a5a0 con 0x564677075860 2024-04-15T15:20:25.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.815+0000 7f68cf400640 1 -- 172.21.15.49:0/4002715922 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f68d804d070 con 0x564677075860 2024-04-15T15:20:25.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.815+0000 7f68f03cd740 10 client.5394 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:20:25.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.815+0000 7f68f03cd740 10 client.5394 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-15T15:20:25.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.815+0000 7f68f03cd740 20 client.5394 populate_metadata read hostname 'smithi049' 2024-04-15T15:20:25.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.815+0000 7f68f03cd740 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x56467706d210 0x56467706d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.815+0000 7f68f03cd740 4 client.5394 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:20:25.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.815+0000 7f68f03cd740 1 -- 172.21.15.49:0/4002715922 --> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564676fc0f60 con 0x56467706d210 2024-04-15T15:20:25.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.815+0000 7f68e6000640 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x56467706d210 0x56467706d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.816+0000 7f68e6000640 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x56467706d210 0x56467706d670 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a143ea2e616ce3 server_cookie=d6048222abe8f323 in_seq=0 out_seq=0 2024-04-15T15:20:25.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.816+0000 7f68cf400640 10 client.5394 ms_handle_connect on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:25.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68cf400640 1 -- 172.21.15.49:0/4002715922 <== mds.0 v2:172.21.15.49:6836/1023024717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564676fc0f60 con 0x56467706d210 2024-04-15T15:20:25.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68cf400640 10 client.5394 handle_command_reply: tid=0 2024-04-15T15:20:25.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 client.5394 shutdown 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 -- 172.21.15.49:0/4002715922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 msgr2=0x56467706e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x56467706e740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009080 tx=0x7f68d802e000 comp rx=0 tx=0).stop 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 -- 172.21.15.49:0/4002715922 shutdown_connections 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x56467706d210 0x56467706d670 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x5646770a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646770a4100 0x564677077cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 --2- 172.21.15.49:0/4002715922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564677075860 0x56467706e740 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68cf400640 0 client.5394 ms_handle_reset on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 -- 172.21.15.49:0/4002715922 >> 172.21.15.49:0/4002715922 conn(0x564677048d40 msgr2=0x564677073d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 -- 172.21.15.49:0/4002715922 shutdown_connections 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 1 -- 172.21.15.49:0/4002715922 wait complete. 2024-04-15T15:20:25.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.817+0000 7f68f03cd740 20 client.5394 trim_cache size 0 max 16384 2024-04-15T15:20:25.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.837+0000 7f68f03cd740 1 Processor -- start 2024-04-15T15:20:25.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.837+0000 7f68f03cd740 1 -- start start 2024-04-15T15:20:25.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.838+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 0x5646770970d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.838+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x56467707b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.838+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.838+0000 7f68f03cd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68e8201030 con 0x5646770a4100 2024-04-15T15:20:25.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.838+0000 7f68f03cd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f68e814e230 con 0x564677075860 2024-04-15T15:20:25.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.838+0000 7f68f03cd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f68e814d590 con 0x564677075c40 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.838+0000 7f68e7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.838+0000 7f68e7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43564/0 (socket says 172.21.15.49:43564) 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.839+0000 7f68e7400640 1 -- 172.21.15.49:0/1735841818 learned_addr learned my addr 172.21.15.49:0/1735841818 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.839+0000 7f68e6000640 1 --2- 172.21.15.49:0/1735841818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x56467707b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.839+0000 7f68e6a00640 1 --2- 172.21.15.49:0/1735841818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 0x5646770970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.839+0000 7f68e7400640 1 -- 172.21.15.49:0/1735841818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 msgr2=0x56467707b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.839+0000 7f68e7400640 1 --2- 172.21.15.49:0/1735841818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x56467707b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.839+0000 7f68e7400640 1 -- 172.21.15.49:0/1735841818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 msgr2=0x5646770970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.839+0000 7f68e7400640 1 --2- 172.21.15.49:0/1735841818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 0x5646770970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.839+0000 7f68e7400640 1 -- 172.21.15.49:0/1735841818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646770a3f30 con 0x5646770a4100 2024-04-15T15:20:25.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.839+0000 7f68e7400640 1 --2- 172.21.15.49:0/1735841818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093b50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f68e0059780 tx=0x7f68e005a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc25ff169487f3f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68d6600640 1 -- 172.21.15.49:0/1735841818 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0088070 con 0x5646770a4100 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68d6600640 1 -- 172.21.15.49:0/1735841818 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68e00623d0 con 0x5646770a4100 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68d6600640 1 -- 172.21.15.49:0/1735841818 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e00600b0 con 0x5646770a4100 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68f03cd740 1 -- 172.21.15.49:0/1735841818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 msgr2=0x564677093b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68f03cd740 1 --2- 172.21.15.49:0/1735841818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093b50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f68e0059780 tx=0x7f68e005a030 comp rx=0 tx=0).stop 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68f03cd740 1 -- 172.21.15.49:0/1735841818 shutdown_connections 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68f03cd740 1 --2- 172.21.15.49:0/1735841818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x56467707b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68f03cd740 1 --2- 172.21.15.49:0/1735841818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093b50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68f03cd740 1 --2- 172.21.15.49:0/1735841818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 0x5646770970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68f03cd740 1 -- 172.21.15.49:0/1735841818 >> 172.21.15.49:0/1735841818 conn(0x564677048d40 msgr2=0x56467704f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68f03cd740 1 -- 172.21.15.49:0/1735841818 shutdown_connections 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.840+0000 7f68f03cd740 1 -- 172.21.15.49:0/1735841818 wait complete. 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.841+0000 7f68f03cd740 1 -- start start 2024-04-15T15:20:25.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.841+0000 7f68f03cd740 1 Processor -- start 2024-04-15T15:20:25.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.842+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 0x5646770f6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.842+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x5646770939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.842+0000 7f68f03cd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.842+0000 7f68f03cd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68e814dbe0 con 0x5646770a4100 2024-04-15T15:20:25.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.842+0000 7f68f03cd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564677049e40 con 0x564677075860 2024-04-15T15:20:25.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.842+0000 7f68f03cd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x5646770a2740 con 0x564677075c40 2024-04-15T15:20:25.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68f03cd740 10 client.0 _ll_register_callbacks cb 0x7f68d8051070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43574/0 (socket says 172.21.15.49:43574) 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e7400640 1 -- 172.21.15.49:0/1890354639 learned_addr learned my addr 172.21.15.49:0/1890354639 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e6a00640 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 0x5646770f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e6000640 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x5646770939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e7400640 1 -- 172.21.15.49:0/1890354639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 msgr2=0x5646770939c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e7400640 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x5646770939c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e7400640 1 -- 172.21.15.49:0/1890354639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 msgr2=0x5646770f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e7400640 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 0x5646770f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e7400640 1 -- 172.21.15.49:0/1890354639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564677070330 con 0x5646770a4100 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.843+0000 7f68e6a00640 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 0x5646770f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68e7400640 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093f00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f68e00594f0 tx=0x7f68e001c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352a8e58a52528ea server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68cf400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-15T15:20:25.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68cf400640 1 -- 172.21.15.49:0/1890354639 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e007b070 con 0x5646770a4100 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68cf400640 1 -- 172.21.15.49:0/1890354639 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f68e0058c40 con 0x5646770a4100 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68f03cd740 1 -- 172.21.15.49:0/1890354639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68e00623d0 con 0x5646770a4100 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68cf400640 1 -- 172.21.15.49:0/1890354639 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f68e0077070 con 0x5646770a4100 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68f03cd740 10 client.5400 fetch_fsmap learned FSMap version 17 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68f03cd740 1 -- 172.21.15.49:0/1890354639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x564676c85920 con 0x5646770a4100 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68cf400640 1 -- 172.21.15.49:0/1890354639 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68e00623d0 con 0x5646770a4100 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68cf400640 1 -- 172.21.15.49:0/1890354639 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f68e007e030 con 0x5646770a4100 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68f03cd740 10 client.5400 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68f03cd740 10 client.5400 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.844+0000 7f68f03cd740 20 client.5400 populate_metadata read hostname 'smithi049' 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.845+0000 7f68f03cd740 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x564677071290 0x564677073910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.845+0000 7f68f03cd740 4 client.5400 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "quiesce path", "path": "/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b", "wait": true}] 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.845+0000 7f68f03cd740 1 -- 172.21.15.49:0/1890354639 --> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] -- command(tid 0: {"prefix": "quiesce path", "path": "/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b", "wait": true}) -- 0x564676fbadd0 con 0x564677071290 2024-04-15T15:20:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.845+0000 7f68e6a00640 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x564677071290 0x564677073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.845+0000 7f68e6a00640 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x564677071290 0x564677073910 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa0a1eabd4a15bae server_cookie=dd93b0f3204134a9 in_seq=0 out_seq=0 2024-04-15T15:20:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.845+0000 7f68cf400640 10 client.5400 ms_handle_connect on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68cf400640 1 -- 172.21.15.49:0/1890354639 <== mds.0 v2:172.21.15.49:6836/1023024717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2332 (crc 0 0 0) 0x564676fbadd0 con 0x564677071290 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68cf400640 10 client.5400 handle_command_reply: tid=0 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68f03cd740 1 client.5400 shutdown 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68f03cd740 1 -- 172.21.15.49:0/1890354639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 msgr2=0x564677093f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68f03cd740 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093f00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f68e00594f0 tx=0x7f68e001c070 comp rx=0 tx=0).stop 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68f03cd740 1 -- 172.21.15.49:0/1890354639 shutdown_connections 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68f03cd740 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x564677071290 0x564677073910 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68f03cd740 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564677075c40 0x5646770939c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68cf400640 0 client.5400 ms_handle_reset on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68f03cd740 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646770a4100 0x564677093f00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68f03cd740 1 --2- 172.21.15.49:0/1890354639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564677075860 0x5646770f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.851+0000 7f68f03cd740 1 -- 172.21.15.49:0/1890354639 >> 172.21.15.49:0/1890354639 conn(0x564677048d40 msgr2=0x56467704e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.852+0000 7f68f03cd740 1 -- 172.21.15.49:0/1890354639 shutdown_connections 2024-04-15T15:20:25.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.852+0000 7f68f03cd740 1 -- 172.21.15.49:0/1890354639 wait complete. 2024-04-15T15:20:25.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.852+0000 7f68f03cd740 20 client.5400 trim_cache size 0 max 16384 2024-04-15T15:20:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "op": { 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "result": 0, 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "quiesce complete", 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "reqid": { 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": { 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "mds", 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 0 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 1 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "op_type": "internal_op", 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "internal_op": 5384, 2024-04-15T15:20:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: "op_name": "quiesce_path", 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.847834+0000", 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated" 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.847834+0000", 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read" 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.847834+0000", 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled" 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.847834+0000", 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read" 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.847834+0000", 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched" 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.847939+0000", 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "acquired locks" 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.847963+0000", 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "acquired locks" 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.847985+0000", 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "acquired locks" 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.848007+0000", 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "acquired locks" 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.852739+0000", 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "acquired locks" 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.852756+0000", 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "acquired locks" 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.852792+0000", 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "acquired locks" 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.852817+0000", 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "acquired locks" 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-15T15:20:25.852840+0000", 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "quiesce complete" 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-15T15:20:25.860 INFO:teuthology.orchestra.run.smithi049.stdout: "locks": null 2024-04-15T15:20:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-15T15:20:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "state": { 2024-04-15T15:20:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 2, 2024-04-15T15:20:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_quiesced": 1, 2024-04-15T15:20:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_blocked": 1, 2024-04-15T15:20:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_dropped": 0, 2024-04-15T15:20:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: "failed": [] 2024-04-15T15:20:25.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-15T15:20:25.861 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:20:25.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.852+0000 7f68e7e00640 1 -- 172.21.15.49:0/2888099332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f68c406c9b0 msgr2=0x7f68c406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.852+0000 7f68e7e00640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f68c406c9b0 0x7f68c406ee70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f68d8036000 tx=0x7f68d803c040 comp rx=0 tx=0).stop 2024-04-15T15:20:25.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.852+0000 7f68e7e00640 1 -- 172.21.15.49:0/2888099332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80d2b90 msgr2=0x7f68e81e7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:25.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.852+0000 7f68e7e00640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80d2b90 0x7f68e81e7880 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f68e000b410 tx=0x7f68e000b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.853+0000 7f68e7e00640 1 -- 172.21.15.49:0/2888099332 shutdown_connections 2024-04-15T15:20:25.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.853+0000 7f68e7e00640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f68c406c9b0 0x7f68c406ee70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.853+0000 7f68e7e00640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e80d2b90 0x7f68e81e7880 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.853+0000 7f68e7e00640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e80cf510 0x7f68e81e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.853+0000 7f68e7e00640 1 --2- 172.21.15.49:0/2888099332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e80b78e0 0x7f68e80b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:25.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.853+0000 7f68e7e00640 1 -- 172.21.15.49:0/2888099332 >> 172.21.15.49:0/2888099332 conn(0x7f68e80d13a0 msgr2=0x7f68e80c8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:25.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.853+0000 7f68e7e00640 1 -- 172.21.15.49:0/2888099332 shutdown_connections 2024-04-15T15:20:25.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:25.853+0000 7f68e7e00640 1 -- 172.21.15.49:0/2888099332 wait complete. 2024-04-15T15:20:25.874 DEBUG:tasks.cephfs.test_quiesce:{'op': {'result': 0, 'flag_point': 'quiesce complete', 'reqid': {'entity': {'type': 'mds', 'num': 0}, 'tid': 1}, 'op_type': 'internal_op', 'internal_op': 5384, 'op_name': 'quiesce_path', 'events': [{'time': '2024-04-15T15:20:25.847834+0000', 'event': 'initiated'}, {'time': '2024-04-15T15:20:25.847834+0000', 'event': 'header_read'}, {'time': '2024-04-15T15:20:25.847834+0000', 'event': 'throttled'}, {'time': '2024-04-15T15:20:25.847834+0000', 'event': 'all_read'}, {'time': '2024-04-15T15:20:25.847834+0000', 'event': 'dispatched'}, {'time': '2024-04-15T15:20:25.847939+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:20:25.847963+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:20:25.847985+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:20:25.848007+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:20:25.852739+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:20:25.852756+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:20:25.852792+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:20:25.852817+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:20:25.852840+0000', 'event': 'quiesce complete'}], 'locks': None}, 'state': {'inodes': 2, 'inodes_quiesced': 1, 'inodes_blocked': 1, 'inodes_dropped': 0, 'failed': []}} 2024-04-15T15:20:25.874 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:26.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.094+0000 7fdabbe00640 1 Processor -- start 2024-04-15T15:20:26.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.095+0000 7fdabbe00640 1 -- start start 2024-04-15T15:20:26.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.096+0000 7fdabbe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdabc162020 0x7fdabc162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.096+0000 7fdabbe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdabc0cf110 0x7fdabc0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.096+0000 7fdabbe00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdabc0d0090 con 0x7fdabc0d1ac0 2024-04-15T15:20:26.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.096+0000 7fdabbe00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fdabc0d1f50 con 0x7fdabc162020 2024-04-15T15:20:26.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.096+0000 7fdabbe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdabc163440 con 0x7fdabc0cf110 2024-04-15T15:20:26.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.096+0000 7fdabaa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdabc162020 0x7fdabc162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.096+0000 7fdabaa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdabc162020 0x7fdabc162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43584/0 (socket says 172.21.15.49:43584) 2024-04-15T15:20:26.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.096+0000 7fdabaa00640 1 -- 172.21.15.49:0/3679138661 learned_addr learned my addr 172.21.15.49:0/3679138661 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.096+0000 7fdabb400640 1 --1- 172.21.15.49:0/3679138661 >> v1:172.21.15.49:6789/0 conn(0x7fdabc0d1ac0 0x7fdabc0b62a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42172/0 (socket says 172.21.15.49:42172) 2024-04-15T15:20:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.097+0000 7fdabaa00640 1 -- 172.21.15.49:0/3679138661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdabc0cf110 msgr2=0x7fdabc0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.097+0000 7fdabaa00640 1 --2- 172.21.15.49:0/3679138661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdabc0cf110 0x7fdabc0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.097+0000 7fdabaa00640 1 -- 172.21.15.49:0/3679138661 >> v1:172.21.15.49:6789/0 conn(0x7fdabc0d1ac0 legacy=0x7fdabc0b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.097+0000 7fdabaa00640 1 -- 172.21.15.49:0/3679138661 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc0cfad0 con 0x7fdabc162020 2024-04-15T15:20:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.098+0000 7fdabaa00640 1 --2- 172.21.15.49:0/3679138661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc162020 0x7fdabc162f00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009870 tx=0x7fdaac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4c98d27e58a893 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.098+0000 7fdab9600640 1 -- 172.21.15.49:0/3679138661 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac034070 con 0x7fdabc162020 2024-04-15T15:20:26.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.098+0000 7fdab9600640 1 -- 172.21.15.49:0/3679138661 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaac030840 con 0x7fdabc162020 2024-04-15T15:20:26.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.098+0000 7fdab9600640 1 -- 172.21.15.49:0/3679138661 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac030c10 con 0x7fdabc162020 2024-04-15T15:20:26.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.099+0000 7fdabbe00640 1 -- 172.21.15.49:0/3679138661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc162020 msgr2=0x7fdabc162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.099+0000 7fdabbe00640 1 --2- 172.21.15.49:0/3679138661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc162020 0x7fdabc162f00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009870 tx=0x7fdaac02b980 comp rx=0 tx=0).stop 2024-04-15T15:20:26.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.099+0000 7fdabbe00640 1 -- 172.21.15.49:0/3679138661 shutdown_connections 2024-04-15T15:20:26.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.099+0000 7fdabbe00640 1 --2- 172.21.15.49:0/3679138661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdabc0cf110 0x7fdabc0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.099+0000 7fdabbe00640 1 --2- 172.21.15.49:0/3679138661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc162020 0x7fdabc162f00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.099+0000 7fdabbe00640 1 -- 172.21.15.49:0/3679138661 >> 172.21.15.49:0/3679138661 conn(0x7fdabc06cf00 msgr2=0x7fdabc06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.099+0000 7fdabbe00640 1 -- 172.21.15.49:0/3679138661 shutdown_connections 2024-04-15T15:20:26.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.099+0000 7fdabbe00640 1 -- 172.21.15.49:0/3679138661 wait complete. 2024-04-15T15:20:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.101+0000 7fdabbe00640 1 Processor -- start 2024-04-15T15:20:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.101+0000 7fdabbe00640 1 -- start start 2024-04-15T15:20:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.102+0000 7fdabbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc0cf110 0x7fdabc1573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.102+0000 7fdabbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdabc0d1ac0 0x7fdabc160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.102+0000 7fdabbe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdabc162020 0x7fdabc160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.102+0000 7fdabbe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc0cf880 con 0x7fdabc0cf110 2024-04-15T15:20:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.102+0000 7fdabbe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc0b7b30 con 0x7fdabc0d1ac0 2024-04-15T15:20:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.102+0000 7fdabbe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdabc0d2b90 con 0x7fdabc162020 2024-04-15T15:20:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.102+0000 7fdabaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc0cf110 0x7fdabc1573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.102+0000 7fdabaa00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc0cf110 0x7fdabc1573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43596/0 (socket says 172.21.15.49:43596) 2024-04-15T15:20:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.102+0000 7fdabaa00640 1 -- 172.21.15.49:0/4246243183 learned_addr learned my addr 172.21.15.49:0/4246243183 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.103+0000 7fdabaa00640 1 -- 172.21.15.49:0/4246243183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdabc162020 msgr2=0x7fdabc160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.103+0000 7fdabaa00640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdabc162020 0x7fdabc160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.103+0000 7fdabaa00640 1 -- 172.21.15.49:0/4246243183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdabc0d1ac0 msgr2=0x7fdabc160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.103+0000 7fdabaa00640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdabc0d1ac0 0x7fdabc160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.103+0000 7fdabaa00640 1 -- 172.21.15.49:0/4246243183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc1e8cb0 con 0x7fdabc0cf110 2024-04-15T15:20:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.103+0000 7fdabaa00640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc0cf110 0x7fdabc1573c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdaac000c00 tx=0x7fdaac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.103+0000 7fdab2a00640 1 -- 172.21.15.49:0/4246243183 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac048070 con 0x7fdabc0cf110 2024-04-15T15:20:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.103+0000 7fdab2a00640 1 -- 172.21.15.49:0/4246243183 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaac030640 con 0x7fdabc0cf110 2024-04-15T15:20:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.103+0000 7fdabbe00640 1 -- 172.21.15.49:0/4246243183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdabc1e8f30 con 0x7fdabc0cf110 2024-04-15T15:20:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.103+0000 7fdab2a00640 1 -- 172.21.15.49:0/4246243183 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac034040 con 0x7fdabc0cf110 2024-04-15T15:20:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.104+0000 7fdabbe00640 1 -- 172.21.15.49:0/4246243183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdabc1e94f0 con 0x7fdabc0cf110 2024-04-15T15:20:26.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.105+0000 7fdabbe00640 1 -- 172.21.15.49:0/4246243183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdabc1e9bb0 con 0x7fdabc0cf110 2024-04-15T15:20:26.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.106+0000 7fdab2a00640 1 -- 172.21.15.49:0/4246243183 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fdaac04b070 con 0x7fdabc0cf110 2024-04-15T15:20:26.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.107+0000 7fdab2a00640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fda9806c980 0x7fda9806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.108+0000 7fdab2a00640 1 -- 172.21.15.49:0/4246243183 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fdaac0adf10 con 0x7fdabc0cf110 2024-04-15T15:20:26.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.108+0000 7fdaba000640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fda9806c980 0x7fda9806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.108+0000 7fdaba000640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fda9806c980 0x7fda9806ee40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdabc159660 tx=0x7fdaa4009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.112+0000 7fdab2a00640 1 -- 172.21.15.49:0/4246243183 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdabc1e9bb0 con 0x7fdabc0cf110 2024-04-15T15:20:26.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.358+0000 7fdabbe00640 1 -- 172.21.15.49:0/4246243183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdabc0755c0 con 0x7fdabc0cf110 2024-04-15T15:20:26.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.359+0000 7fdab2a00640 1 -- 172.21.15.49:0/4246243183 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7fdabc1e8cb0 con 0x7fdabc0cf110 2024-04-15T15:20:26.362 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:26.362 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:00.037472+0000","modified":"2024-04-15T15:20:03.151587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5067":{"gid":5067,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/1023024717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1023024717},{"type":"v1","addr":"172.21.15.49:6837","nonce":1023024717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5067,"qdb_cluster":[5067]},"id":2}]} 2024-04-15T15:20:26.363 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-04-15T15:20:26.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 -- 172.21.15.49:0/4246243183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fda9806c980 msgr2=0x7fda9806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fda9806c980 0x7fda9806ee40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdabc159660 tx=0x7fdaa4009250 comp rx=0 tx=0).stop 2024-04-15T15:20:26.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 -- 172.21.15.49:0/4246243183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc0cf110 msgr2=0x7fdabc1573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc0cf110 0x7fdabc1573c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdaac000c00 tx=0x7fdaac004730 comp rx=0 tx=0).stop 2024-04-15T15:20:26.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 -- 172.21.15.49:0/4246243183 shutdown_connections 2024-04-15T15:20:26.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fda9806c980 0x7fda9806ee40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdabc162020 0x7fdabc160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdabc0d1ac0 0x7fdabc160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 --2- 172.21.15.49:0/4246243183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdabc0cf110 0x7fdabc1573c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 -- 172.21.15.49:0/4246243183 >> 172.21.15.49:0/4246243183 conn(0x7fdabc06cf00 msgr2=0x7fdabc0b9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 -- 172.21.15.49:0/4246243183 shutdown_connections 2024-04-15T15:20:26.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.362+0000 7fdabbe00640 1 -- 172.21.15.49:0/4246243183 wait complete. 2024-04-15T15:20:26.386 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b 0 2024-04-15T15:20:26.637 INFO:teuthology.orchestra.run.smithi049.stdout:[{"ino":1099511627779,"rdev":0,"ctime":"2024-04-15T15:20:25.443949+0000","btime":"2024-04-15T15:20:12.433914+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":5,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-15T15:20:25.443949+0000","atime":"2024-04-15T15:20:12.433914+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-15T15:20:25.443949+0000","num_files":1,"num_subdirs":0,"change_attr":1},"rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-15T15:20:25.542946+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-15T15:20:25.542946+0000"},"version":22,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":2,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":1},"nref":8,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{"reqid":{"entity":{"type":"mds","num":0},"tid":2}}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5017,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFs","last_sent":8},{"client_id":5023,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFs","last_sent":8},{"client_id":5280,"pending":"pAsLsXs","issued":"pAsLsXs","wanted":"-","last_sent":9},{"client_id":5361,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFsxcral","last_sent":30},{"client_id":5370,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFs","last_sent":8}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/pyvenv.cfg","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":1,"projected_version":0,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"lock":0,"authpin":0},"nref":0,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth"]},{"path":"volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/file","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629282,"remote_type":"","version":6,"projected_version":6,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-15T15:20:26.639 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-04-15T15:20:25.542946+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-15T15:20:12.433914+0000", "auth_pins": 1, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-15T15:20:12.433914+0000", "change_attr": 3, "client_caps": [ { "client_id": 5017, "issued": "pAsLsXsFs", "last_sent": 8, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFs" }, { "client_id": 5023, "issued": "pAsLsXsFs", "last_sent": 8, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFs" }, { "client_id": 5280, "issued": "pAsLsXs", "last_sent": 9, "pending": "pAsLsXs", "wanted": "-" }, { "client_id": 5361, "issued": "pAsLsXsFs", "last_sent": 30, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFsxcral" }, { "client_id": 5370, "issued": "pAsLsXsFs", "last_sent": 8, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFs" } ], "client_ranges": [], "ctime": "2024-04-15T15:20:25.443949+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 0, "path": "volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/pyvenv.cfg", "path_ino": 1, "pins": { "authpin": 0, "lock": 0 }, "projected_version": 0, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth" ], "version": 1, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629282, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/file", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 6, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 6, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 1, "mtime": "2024-04-15T15:20:25.443949+0000", "num_files": 1, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1099511627779, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 5, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2024-04-15T15:20:25.443949+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 8, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 2, "request": 1, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "lock", "type": "iquiesce", "xlock_by": { "reqid": { "entity": { "num": 0, "type": "mds" }, "tid": 2 } } }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-04-15T15:20:25.542946+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 22, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-04-15T15:20:26.639 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 ops --flags=locks --path=/tmp/mds.0-ops 2024-04-15T15:20:26.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.871+0000 7feecb200640 1 Processor -- start 2024-04-15T15:20:26.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.871+0000 7feecb200640 1 -- start start 2024-04-15T15:20:26.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.873+0000 7feecb200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7feecc162c00 0x7feecc0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.873+0000 7feecb200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feecc1674f0 con 0x7feecc0b05a0 2024-04-15T15:20:26.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.873+0000 7feecb200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feecc0cc950 con 0x7feecc0725f0 2024-04-15T15:20:26.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.873+0000 7feecb200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7feecc1672a0 con 0x7feecc162c00 2024-04-15T15:20:26.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.873+0000 7feec9e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7feecc162c00 0x7feecc0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.873+0000 7feec9e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7feecc162c00 0x7feecc0d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43610/0 (socket says 172.21.15.49:43610) 2024-04-15T15:20:26.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.873+0000 7feec9e00640 1 -- 172.21.15.49:0/1880633673 learned_addr learned my addr 172.21.15.49:0/1880633673 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:26.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.873+0000 7feec9400640 1 --1- 172.21.15.49:0/1880633673 >> v1:172.21.15.49:6789/0 conn(0x7feecc0b05a0 0x7feecc071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42178/0 (socket says 172.21.15.49:42178) 2024-04-15T15:20:26.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.874+0000 7feec9e00640 1 -- 172.21.15.49:0/1880633673 >> v1:172.21.15.107:6789/0 conn(0x7feecc0725f0 legacy=0x7feecc06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.874+0000 7feec9e00640 1 -- 172.21.15.49:0/1880633673 >> v1:172.21.15.49:6789/0 conn(0x7feecc0b05a0 legacy=0x7feecc071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.874+0000 7feec9e00640 1 -- 172.21.15.49:0/1880633673 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc0ccc30 con 0x7feecc162c00 2024-04-15T15:20:26.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.875+0000 7feec9e00640 1 --2- 172.21.15.49:0/1880633673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc162c00 0x7feecc0d0010 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feebc009870 tx=0x7feebc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ac27d47426fdc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.875+0000 7feec8a00640 1 -- 172.21.15.49:0/1880633673 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc035020 con 0x7feecc162c00 2024-04-15T15:20:26.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.875+0000 7feec8a00640 1 -- 172.21.15.49:0/1880633673 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc031870 con 0x7feecc162c00 2024-04-15T15:20:26.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.875+0000 7feec8a00640 1 -- 172.21.15.49:0/1880633673 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc031c40 con 0x7feecc162c00 2024-04-15T15:20:26.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.876+0000 7feecb200640 1 -- 172.21.15.49:0/1880633673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc162c00 msgr2=0x7feecc0d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.876+0000 7feecb200640 1 --2- 172.21.15.49:0/1880633673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc162c00 0x7feecc0d0010 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feebc009870 tx=0x7feebc02b910 comp rx=0 tx=0).stop 2024-04-15T15:20:26.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.876+0000 7feecb200640 1 -- 172.21.15.49:0/1880633673 shutdown_connections 2024-04-15T15:20:26.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.876+0000 7feecb200640 1 --2- 172.21.15.49:0/1880633673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc162c00 0x7feecc0d0010 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.876+0000 7feecb200640 1 -- 172.21.15.49:0/1880633673 >> 172.21.15.49:0/1880633673 conn(0x7feecc06b930 msgr2=0x7feecc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.876+0000 7feecb200640 1 -- 172.21.15.49:0/1880633673 shutdown_connections 2024-04-15T15:20:26.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.876+0000 7feecb200640 1 -- 172.21.15.49:0/1880633673 wait complete. 2024-04-15T15:20:26.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.878+0000 7feecb200640 1 Processor -- start 2024-04-15T15:20:26.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.878+0000 7feecb200640 1 -- start start 2024-04-15T15:20:26.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.879+0000 7feecb200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feecc0725f0 0x7feecc14b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.879+0000 7feecb200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc0b05a0 0x7feecc160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.879+0000 7feecb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feecc162c00 0x7feecc1611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.879+0000 7feecb200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feecc06eaa0 con 0x7feecc0b05a0 2024-04-15T15:20:26.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.879+0000 7feecb200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1677d0 con 0x7feecc162c00 2024-04-15T15:20:26.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.879+0000 7feecb200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7feecc06ec20 con 0x7feecc0725f0 2024-04-15T15:20:26.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.879+0000 7feec9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc0b05a0 0x7feecc160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.879+0000 7feec9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc0b05a0 0x7feecc160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43620/0 (socket says 172.21.15.49:43620) 2024-04-15T15:20:26.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.879+0000 7feec9400640 1 -- 172.21.15.49:0/627570256 learned_addr learned my addr 172.21.15.49:0/627570256 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:26.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feec9e00640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feecc0725f0 0x7feecc14b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feec9400640 1 -- 172.21.15.49:0/627570256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feecc0725f0 msgr2=0x7feecc14b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feeca800640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feecc162c00 0x7feecc1611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feec9400640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feecc0725f0 0x7feecc14b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feec9400640 1 -- 172.21.15.49:0/627570256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feecc162c00 msgr2=0x7feecc1611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feec9400640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feecc162c00 0x7feecc1611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feec9400640 1 -- 172.21.15.49:0/627570256 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc163610 con 0x7feecc0b05a0 2024-04-15T15:20:26.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feeca800640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feecc162c00 0x7feecc1611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:26.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feec9400640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc0b05a0 0x7feecc160ca0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c400 tx=0x7feeb800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feec2000640 1 -- 172.21.15.49:0/627570256 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80074e0 con 0x7feecc0b05a0 2024-04-15T15:20:26.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.880+0000 7feec2000640 1 -- 172.21.15.49:0/627570256 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8007680 con 0x7feecc0b05a0 2024-04-15T15:20:26.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.881+0000 7feecb200640 1 -- 172.21.15.49:0/627570256 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feecc1659e0 con 0x7feecc0b05a0 2024-04-15T15:20:26.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.881+0000 7feec2000640 1 -- 172.21.15.49:0/627570256 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8004a00 con 0x7feecc0b05a0 2024-04-15T15:20:26.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.881+0000 7feecb200640 1 -- 172.21.15.49:0/627570256 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feecc1638f0 con 0x7feecc0b05a0 2024-04-15T15:20:26.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.884+0000 7feec2000640 1 -- 172.21.15.49:0/627570256 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7feeb801a020 con 0x7feecc0b05a0 2024-04-15T15:20:26.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.885+0000 7feec2000640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feeac06c980 0x7feeac06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.885+0000 7feec2000640 1 -- 172.21.15.49:0/627570256 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7feeb808b350 con 0x7feecc0b05a0 2024-04-15T15:20:26.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.885+0000 7feed3ba5740 1 Processor -- start 2024-04-15T15:20:26.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.885+0000 7feec9e00640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feeac06c980 0x7feeac06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.885+0000 7feed3ba5740 1 -- start start 2024-04-15T15:20:26.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.886+0000 7feec9e00640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feeac06c980 0x7feeac06ee40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feebc036000 tx=0x7feebc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.886+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.886+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c6630 0x55a0ea4c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.886+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 0x55a0ea4c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.886+0000 7feed3ba5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a0ea4037b0 con 0x55a0ea4c9210 2024-04-15T15:20:26.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.886+0000 7feed3ba5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0ea3aa2d0 con 0x55a0ea49e340 2024-04-15T15:20:26.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.886+0000 7feed3ba5740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x55a0ea3ff770 con 0x55a0ea4c6630 2024-04-15T15:20:26.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feec9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feec9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43632/0 (socket says 172.21.15.49:43632) 2024-04-15T15:20:26.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feec9e00640 1 -- 172.21.15.49:0/2185593425 learned_addr learned my addr 172.21.15.49:0/2185593425 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:26.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feeca800640 1 --2- 172.21.15.49:0/2185593425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 0x55a0ea4c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feec9400640 1 --2- 172.21.15.49:0/2185593425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c6630 0x55a0ea4c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feec9e00640 1 -- 172.21.15.49:0/2185593425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c6630 msgr2=0x55a0ea4c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feec9e00640 1 --2- 172.21.15.49:0/2185593425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c6630 0x55a0ea4c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feec9e00640 1 -- 172.21.15.49:0/2185593425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 msgr2=0x55a0ea4c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feec9e00640 1 --2- 172.21.15.49:0/2185593425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 0x55a0ea4c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feec9e00640 1 -- 172.21.15.49:0/2185593425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ea116420 con 0x55a0ea4c9210 2024-04-15T15:20:26.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.887+0000 7feeca800640 1 --2- 172.21.15.49:0/2185593425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 0x55a0ea4c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:26.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.888+0000 7feec9e00640 1 --2- 172.21.15.49:0/2185593425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4a00a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feebc000c00 tx=0x7feebc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3459f50aaa65ed88 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.888+0000 7feeb6a00640 1 -- 172.21.15.49:0/2185593425 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc03aaf0 con 0x55a0ea4c9210 2024-04-15T15:20:26.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.888+0000 7feeb6a00640 1 -- 172.21.15.49:0/2185593425 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc03ac90 con 0x55a0ea4c9210 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.888+0000 7feeb6a00640 1 -- 172.21.15.49:0/2185593425 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc04f5f0 con 0x55a0ea4c9210 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.889+0000 7feed3ba5740 1 -- 172.21.15.49:0/2185593425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 msgr2=0x55a0ea4a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.889+0000 7feed3ba5740 1 --2- 172.21.15.49:0/2185593425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4a00a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feebc000c00 tx=0x7feebc030000 comp rx=0 tx=0).stop 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.889+0000 7feed3ba5740 1 -- 172.21.15.49:0/2185593425 shutdown_connections 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.889+0000 7feed3ba5740 1 --2- 172.21.15.49:0/2185593425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c6630 0x55a0ea4c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.889+0000 7feed3ba5740 1 --2- 172.21.15.49:0/2185593425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 0x55a0ea4c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.889+0000 7feed3ba5740 1 --2- 172.21.15.49:0/2185593425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4a00a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.889+0000 7feed3ba5740 1 -- 172.21.15.49:0/2185593425 >> 172.21.15.49:0/2185593425 conn(0x55a0ea433f80 msgr2=0x55a0ea4e4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.889+0000 7feed3ba5740 1 -- 172.21.15.49:0/2185593425 shutdown_connections 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.889+0000 7feed3ba5740 1 -- 172.21.15.49:0/2185593425 wait complete. 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.890+0000 7feed3ba5740 1 -- start start 2024-04-15T15:20:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.890+0000 7feed3ba5740 1 Processor -- start 2024-04-15T15:20:26.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.891+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea49e340 0x55a0ea4cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.891+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 0x55a0ea4bce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.891+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 0x55a0ea4c3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.891+0000 7feed3ba5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a0ea499640 con 0x55a0ea49e340 2024-04-15T15:20:26.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.891+0000 7feed3ba5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0ea3bd6a0 con 0x55a0ea4c6630 2024-04-15T15:20:26.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.891+0000 7feed3ba5740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x55a0ea4e47d0 con 0x55a0ea4c9210 2024-04-15T15:20:26.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.891+0000 7feed3ba5740 10 client.0 _ll_register_callbacks cb 0x55a0ea3c1f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.891+0000 7feec9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea49e340 0x55a0ea4cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.891+0000 7feec9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea49e340 0x55a0ea4cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43644/0 (socket says 172.21.15.49:43644) 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.891+0000 7feec9e00640 1 -- 172.21.15.49:0/3378551308 learned_addr learned my addr 172.21.15.49:0/3378551308 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feec9400640 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 0x55a0ea4bce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feec9e00640 1 -- 172.21.15.49:0/3378551308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 msgr2=0x55a0ea4c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feeca800640 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 0x55a0ea4c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feec9e00640 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 0x55a0ea4c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feec9e00640 1 -- 172.21.15.49:0/3378551308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 msgr2=0x55a0ea4bce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feec9e00640 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 0x55a0ea4bce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feec9e00640 1 -- 172.21.15.49:0/3378551308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ea1e2830 con 0x55a0ea49e340 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feec9400640 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 0x55a0ea4bce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feeca800640 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 0x55a0ea4c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feec9e00640 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea49e340 0x55a0ea4cc1e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7feebc0090b0 tx=0x7feebc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb2268ee93cb040 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feeab400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.892+0000 7feeab400640 1 -- 172.21.15.49:0/3378551308 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc003a20 con 0x55a0ea49e340 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feeab400640 1 -- 172.21.15.49:0/3378551308 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7feebc003bc0 con 0x55a0ea49e340 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feed3ba5740 1 -- 172.21.15.49:0/3378551308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feebc03ac90 con 0x55a0ea49e340 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feeab400640 1 -- 172.21.15.49:0/3378551308 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7feebc003dc0 con 0x55a0ea49e340 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feed3ba5740 10 client.5424 fetch_fsmap learned FSMap version 17 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feed3ba5740 1 -- 172.21.15.49:0/3378551308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a0ea4c39c0 con 0x55a0ea49e340 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feeab400640 1 -- 172.21.15.49:0/3378551308 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc03ac90 con 0x55a0ea49e340 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feeab400640 1 -- 172.21.15.49:0/3378551308 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7feebc057410 con 0x55a0ea49e340 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feed3ba5740 10 client.5424 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feed3ba5740 10 client.5424 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feed3ba5740 20 client.5424 populate_metadata read hostname 'smithi049' 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feed3ba5740 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x55a0ea4ca230 0x55a0ea4ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feed3ba5740 4 client.5424 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:20:26.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.893+0000 7feed3ba5740 1 -- 172.21.15.49:0/3378551308 --> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0ea4111d0 con 0x55a0ea4ca230 2024-04-15T15:20:26.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.894+0000 7feec9400640 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x55a0ea4ca230 0x55a0ea4ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.894+0000 7feec9400640 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x55a0ea4ca230 0x55a0ea4ca6c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c8c1ff95513901 server_cookie=a62831b819123ed0 in_seq=0 out_seq=0 2024-04-15T15:20:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.894+0000 7feeab400640 10 client.5424 ms_handle_connect on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feeab400640 1 -- 172.21.15.49:0/3378551308 <== mds.0 v2:172.21.15.49:6836/1023024717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0ea4111d0 con 0x55a0ea4ca230 2024-04-15T15:20:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feeab400640 10 client.5424 handle_command_reply: tid=0 2024-04-15T15:20:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feed3ba5740 1 client.5424 shutdown 2024-04-15T15:20:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feed3ba5740 1 -- 172.21.15.49:0/3378551308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea49e340 msgr2=0x55a0ea4cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feed3ba5740 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea49e340 0x55a0ea4cc1e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7feebc0090b0 tx=0x7feebc02e000 comp rx=0 tx=0).stop 2024-04-15T15:20:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feed3ba5740 1 -- 172.21.15.49:0/3378551308 shutdown_connections 2024-04-15T15:20:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feed3ba5740 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x55a0ea4ca230 0x55a0ea4ca6c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feed3ba5740 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 0x55a0ea4c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feed3ba5740 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 0x55a0ea4bce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feed3ba5740 1 --2- 172.21.15.49:0/3378551308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea49e340 0x55a0ea4cc1e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feeab400640 0 client.5424 ms_handle_reset on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:26.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feed3ba5740 1 -- 172.21.15.49:0/3378551308 >> 172.21.15.49:0/3378551308 conn(0x55a0ea433f80 msgr2=0x55a0ea49f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.895+0000 7feed3ba5740 1 -- 172.21.15.49:0/3378551308 shutdown_connections 2024-04-15T15:20:26.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.896+0000 7feed3ba5740 1 -- 172.21.15.49:0/3378551308 wait complete. 2024-04-15T15:20:26.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.896+0000 7feed3ba5740 20 client.5424 trim_cache size 0 max 16384 2024-04-15T15:20:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.913+0000 7feed3ba5740 1 Processor -- start 2024-04-15T15:20:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.913+0000 7feed3ba5740 1 -- start start 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.914+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea49e340 0x55a0ea4c2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.914+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 0x55a0ea4cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.914+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.914+0000 7feed3ba5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feecc2052b0 con 0x55a0ea4c9210 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.914+0000 7feed3ba5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feecc2043f0 con 0x55a0ea4c6630 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.914+0000 7feed3ba5740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7feecc0cd880 con 0x55a0ea49e340 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.915+0000 7feeca800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.915+0000 7feeca800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4e3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43656/0 (socket says 172.21.15.49:43656) 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.915+0000 7feeca800640 1 -- 172.21.15.49:0/3315941939 learned_addr learned my addr 172.21.15.49:0/3315941939 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.915+0000 7feec9400640 1 --2- 172.21.15.49:0/3315941939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 0x55a0ea4cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.915+0000 7feec9e00640 1 --2- 172.21.15.49:0/3315941939 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea49e340 0x55a0ea4c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.915+0000 7feeca800640 1 -- 172.21.15.49:0/3315941939 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea49e340 msgr2=0x55a0ea4c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.915+0000 7feeca800640 1 --2- 172.21.15.49:0/3315941939 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea49e340 0x55a0ea4c2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.915+0000 7feeca800640 1 -- 172.21.15.49:0/3315941939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 msgr2=0x55a0ea4cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.915+0000 7feeca800640 1 --2- 172.21.15.49:0/3315941939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 0x55a0ea4cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.915+0000 7feeca800640 1 -- 172.21.15.49:0/3315941939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ea4ccc80 con 0x55a0ea4c9210 2024-04-15T15:20:26.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.916+0000 7feeca800640 1 --2- 172.21.15.49:0/3315941939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4e3a00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7feecc0cee70 tx=0x7feec4002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ead2a161a981f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.916+0000 7feeb6000640 1 -- 172.21.15.49:0/3315941939 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4003d80 con 0x55a0ea4c9210 2024-04-15T15:20:26.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.916+0000 7feeb6000640 1 -- 172.21.15.49:0/3315941939 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec4018070 con 0x55a0ea4c9210 2024-04-15T15:20:26.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.916+0000 7feeb6000640 1 -- 172.21.15.49:0/3315941939 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec400fb40 con 0x55a0ea4c9210 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.916+0000 7feed3ba5740 1 -- 172.21.15.49:0/3315941939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 msgr2=0x55a0ea4e3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.916+0000 7feed3ba5740 1 --2- 172.21.15.49:0/3315941939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4e3a00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7feecc0cee70 tx=0x7feec4002120 comp rx=0 tx=0).stop 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.917+0000 7feed3ba5740 1 -- 172.21.15.49:0/3315941939 shutdown_connections 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.917+0000 7feed3ba5740 1 --2- 172.21.15.49:0/3315941939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c9210 0x55a0ea4e3a00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.917+0000 7feed3ba5740 1 --2- 172.21.15.49:0/3315941939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea4c6630 0x55a0ea4cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.917+0000 7feed3ba5740 1 --2- 172.21.15.49:0/3315941939 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea49e340 0x55a0ea4c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.917+0000 7feed3ba5740 1 -- 172.21.15.49:0/3315941939 >> 172.21.15.49:0/3315941939 conn(0x55a0ea433f80 msgr2=0x55a0ea494820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.917+0000 7feed3ba5740 1 -- 172.21.15.49:0/3315941939 shutdown_connections 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.917+0000 7feed3ba5740 1 -- 172.21.15.49:0/3315941939 wait complete. 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.917+0000 7feed3ba5740 1 -- start start 2024-04-15T15:20:26.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.918+0000 7feed3ba5740 1 Processor -- start 2024-04-15T15:20:26.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 0x55a0ea498c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c6630 0x55a0ea4e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feed3ba5740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 0x55a0ea4ca610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feed3ba5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feecc0cded0 con 0x55a0ea4c6630 2024-04-15T15:20:26.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feed3ba5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0ea542e40 con 0x55a0ea49e340 2024-04-15T15:20:26.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feed3ba5740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x55a0ea4e4250 con 0x55a0ea4c9210 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feed3ba5740 10 client.0 _ll_register_callbacks cb 0x7feebc031b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feec9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c6630 0x55a0ea4e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feec9400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c6630 0x55a0ea4e3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43670/0 (socket says 172.21.15.49:43670) 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feec9400640 1 -- 172.21.15.49:0/576971275 learned_addr learned my addr 172.21.15.49:0/576971275 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feec9e00640 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 0x55a0ea498c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feeca800640 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 0x55a0ea4ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feec9400640 1 -- 172.21.15.49:0/576971275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 msgr2=0x55a0ea4ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feec9400640 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 0x55a0ea4ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feec9400640 1 -- 172.21.15.49:0/576971275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 msgr2=0x55a0ea498c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feec9400640 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 0x55a0ea498c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feec9400640 1 -- 172.21.15.49:0/576971275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ea4e44a0 con 0x55a0ea4c6630 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.919+0000 7feeca800640 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 0x55a0ea4ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:26.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.920+0000 7feec9400640 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c6630 0x55a0ea4e3a00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feeb8001eb0 tx=0x7feeb80574e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1813a4050548d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.920+0000 7feeab400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.920+0000 7feeab400640 1 -- 172.21.15.49:0/576971275 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb805d130 con 0x55a0ea4c6630 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.920+0000 7feeab400640 1 -- 172.21.15.49:0/576971275 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7feeb8059380 con 0x55a0ea4c6630 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.920+0000 7feed3ba5740 1 -- 172.21.15.49:0/576971275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec4018070 con 0x55a0ea4c6630 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.920+0000 7feeab400640 1 -- 172.21.15.49:0/576971275 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7feeb805d2d0 con 0x55a0ea4c6630 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.920+0000 7feed3ba5740 10 client.5430 fetch_fsmap learned FSMap version 17 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.920+0000 7feed3ba5740 1 -- 172.21.15.49:0/576971275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a0ea4e3f40 con 0x55a0ea4c6630 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.920+0000 7feeab400640 1 -- 172.21.15.49:0/576971275 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec4018070 con 0x55a0ea4c6630 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.921+0000 7feeab400640 1 -- 172.21.15.49:0/576971275 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7feeb8061870 con 0x55a0ea4c6630 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.921+0000 7feed3ba5740 10 client.5430 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.921+0000 7feed3ba5740 10 client.5430 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.921+0000 7feed3ba5740 20 client.5430 populate_metadata read hostname 'smithi049' 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.921+0000 7feed3ba5740 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x55a0ea541bb0 0x55a0ea541fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:26.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.921+0000 7feed3ba5740 4 client.5430 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.0-ops"}] 2024-04-15T15:20:26.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.921+0000 7feed3ba5740 1 -- 172.21.15.49:0/576971275 --> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.0-ops"}) -- 0x55a0ea40b040 con 0x55a0ea541bb0 2024-04-15T15:20:26.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.921+0000 7feec9e00640 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x55a0ea541bb0 0x55a0ea541fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:26.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.922+0000 7feec9e00640 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x55a0ea541bb0 0x55a0ea541fc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d579007b203c8003 server_cookie=e46d0b79d4683a10 in_seq=0 out_seq=0 2024-04-15T15:20:26.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.922+0000 7feeab400640 10 client.5430 ms_handle_connect on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:26.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.922+0000 7feeab400640 1 -- 172.21.15.49:0/576971275 <== mds.0 v2:172.21.15.49:6836/1023024717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+33 (crc 0 0 0) 0x55a0ea40b040 con 0x55a0ea541bb0 2024-04-15T15:20:26.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.922+0000 7feeab400640 10 client.5430 handle_command_reply: tid=0 2024-04-15T15:20:26.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.922+0000 7feed3ba5740 1 client.5430 shutdown 2024-04-15T15:20:26.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 1 -- 172.21.15.49:0/576971275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c6630 msgr2=0x55a0ea4e3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c6630 0x55a0ea4e3a00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feeb8001eb0 tx=0x7feeb80574e0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 1 -- 172.21.15.49:0/576971275 shutdown_connections 2024-04-15T15:20:26.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.49:6836/1023024717,v1:172.21.15.49:6837/1023024717] conn(0x55a0ea541bb0 0x55a0ea541fc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55a0ea4c9210 0x55a0ea4ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0ea4c6630 0x55a0ea4e3a00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 1 --2- 172.21.15.49:0/576971275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0ea49e340 0x55a0ea498c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feeab400640 0 client.5430 ms_handle_reset on v2:172.21.15.49:6836/1023024717 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 1 -- 172.21.15.49:0/576971275 >> 172.21.15.49:0/576971275 conn(0x55a0ea433f80 msgr2=0x55a0ea4c24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 1 -- 172.21.15.49:0/576971275 shutdown_connections 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 1 -- 172.21.15.49:0/576971275 wait complete. 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.923+0000 7feed3ba5740 20 client.5430 trim_cache size 0 max 16384 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/tmp/mds.0-ops" 2024-04-15T15:20:26.929 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:20:26.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 -- 172.21.15.49:0/627570256 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feeac06c980 msgr2=0x7feeac06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feeac06c980 0x7feeac06ee40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feebc036000 tx=0x7feebc03c040 comp rx=0 tx=0).stop 2024-04-15T15:20:26.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 -- 172.21.15.49:0/627570256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc0b05a0 msgr2=0x7feecc160ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc0b05a0 0x7feecc160ca0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c400 tx=0x7feeb800c8c0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 -- 172.21.15.49:0/627570256 shutdown_connections 2024-04-15T15:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7feeac06c980 0x7feeac06ee40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feecc162c00 0x7feecc1611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feecc0b05a0 0x7feecc160ca0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 --2- 172.21.15.49:0/627570256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feecc0725f0 0x7feecc14b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 -- 172.21.15.49:0/627570256 >> 172.21.15.49:0/627570256 conn(0x7feecc06b930 msgr2=0x7feecc06e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 -- 172.21.15.49:0/627570256 shutdown_connections 2024-04-15T15:20:26.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:26.924+0000 7feecb200640 1 -- 172.21.15.49:0/627570256 wait complete. 2024-04-15T15:20:26.944 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-04-15T15:20:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.fQvmSgPpjx 2024-04-15T15:20:26.995 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /tmp/mds.0-ops /tmp/tmp.fQvmSgPpjx 2024-04-15T15:20:27.093 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.fQvmSgPpjx 2024-04-15T15:20:27.267 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.fQvmSgPpjx is 8KB 2024-04-15T15:20:27.333 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.fQvmSgPpjx 2024-04-15T15:20:27.388 DEBUG:tasks.cephfs.filesystem:read 7822B of ops 2024-04-15T15:20:27.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b --path=/tmp/mds.0-cache 2024-04-15T15:20:27.645 INFO:teuthology.orchestra.run.smithi049.stdout:{"path":"/tmp/mds.0-cache"} 2024-04-15T15:20:27.646 DEBUG:tasks.cephfs.filesystem:_json_asok output { "path": "/tmp/mds.0-cache" } 2024-04-15T15:20:27.646 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-04-15T15:20:27.701 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.DvYAhLpP3c 2024-04-15T15:20:27.702 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /tmp/mds.0-cache /tmp/tmp.DvYAhLpP3c 2024-04-15T15:20:27.808 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.DvYAhLpP3c 2024-04-15T15:20:27.981 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.DvYAhLpP3c is 7KB 2024-04-15T15:20:28.049 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.DvYAhLpP3c 2024-04-15T15:20:28.108 DEBUG:tasks.cephfs.filesystem:read 7619B of cache 2024-04-15T15:20:28.109 INFO:tasks.cephfs.test_quiesce:qp = 1; qi = 1; qib = 1; qina = 0 2024-04-15T15:20:28.109 ERROR:tasks.cephfs.test_quiesce:inode: { "ino": 1099511627779, "rdev": 0, "ctime": "2024-04-15T15:20:25.443949+0000", "btime": "2024-04-15T15:20:12.433914+0000", "mode": 17407, "uid": 0, "gid": 0, "nlink": 1, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "layout": { "stripe_unit": 4194304, "stripe_count": 1, "object_size": 4194304, "pool_id": 5, "pool_ns": "" }, "old_pools": [], "size": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "truncate_from": 0, "truncate_pending": 0, "mtime": "2024-04-15T15:20:25.443949+0000", "atime": "2024-04-15T15:20:12.433914+0000", "time_warp_seq": 0, "change_attr": 3, "export_pin": -1, "export_ephemeral_random_pin": 0, "export_ephemeral_distributed_pin": false, "quiesce_block": false, "client_ranges": [], "dirstat": { "version": 0, "mtime": "2024-04-15T15:20:25.443949+0000", "num_files": 1, "num_subdirs": 0, "change_attr": 1 }, "rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:20:25.542946+0000" }, "accounted_rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:20:25.542946+0000" }, "version": 22, "file_data_version": 0, "xattr_version": 1, "backtrace_version": 2, "stray_prior_path": "", "max_size_ever": 0, "quota": { "max_bytes": 0, "max_files": 0 }, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "symlink": "", "xattrs": [], "dirfragtree": { "splits": [] }, "old_inodes": [], "oldest_snap": 18446744073709551614, "damage_flags": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8, "versionlock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "authlock": {}, "linklock": {}, "dirfragtreelock": {}, "filelock": {}, "xattrlock": {}, "snaplock": {}, "nestlock": { "gather_set": [], "state": "lock", "type": "inest", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "flocklock": {}, "policylock": {}, "quiescelock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "client_caps": [ { "client_id": 5017, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 }, { "client_id": 5023, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 }, { "client_id": 5280, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 9 }, { "client_id": 5361, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFsxcral", "last_sent": 30 }, { "client_id": 5370, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 } ], "loner": -1, "want_loner": -1, "mds_caps_wanted": [], "dirfrags": [ { "path": "/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b", "dirfrag": "0x10000000003", "snapid_first": 2, "projected_version": "7", "version": "7", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "dentries": [ { "path": "volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/pyvenv.cfg", "path_ino": 1, "snap_first": 2, "snap_last": 18446744073709551614, "is_primary": false, "is_remote": false, "is_null": true, "is_new": false, "inode": 0, "remote_type": "", "version": 1, "projected_version": 0, "auth_pins": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "is_frozen": false, "is_freezing": false, "pins": { "lock": 0, "authpin": 0 }, "nref": 0, "lock": {}, "versionlock": { "gather_set": [], "state": "lock", "type": "dversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth" ] }, { "path": "volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/file", "path_ino": 1, "snap_first": 2, "snap_last": 18446744073709551614, "is_primary": true, "is_remote": false, "is_null": false, "is_new": false, "inode": 1099511629282, "remote_type": "", "version": 6, "projected_version": 6, "auth_pins": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "is_frozen": false, "is_freezing": false, "pins": { "request": 0, "lock": 0, "inodepin": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "lock": {}, "versionlock": { "gather_set": [], "state": "lock", "type": "dversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth", "dirty" ] } ] } ], "is_subvolume": false } 2024-04-15T15:20:28.110 ERROR:tasks.cephfs.test_quiesce:op: { "description": "internal op quiesce_inode:mds.0:2 fp=#0x10000000003", "initiated_at": "2024-04-15T15:20:25.848020+0000", "age": 1.07623227, "duration": 1.076371371, "continuous": true, "type_data": { "result": -2147483648, "flag_point": "quiesce complete", "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 }, "op_type": "internal_op", "internal_op": 5385, "op_name": "quiesce_inode", "events": [ { "time": "2024-04-15T15:20:25.848020+0000", "event": "initiated" }, { "time": "2024-04-15T15:20:25.848020+0000", "event": "header_read" }, { "time": "2024-04-15T15:20:25.848020+0000", "event": "throttled" }, { "time": "2024-04-15T15:20:25.848020+0000", "event": "all_read" }, { "time": "2024-04-15T15:20:25.848020+0000", "event": "dispatched" }, { "time": "2024-04-15T15:20:25.848429+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:20:25.849067+0000", "event": "acquired locks" }, { "time": "2024-04-15T15:20:25.852708+0000", "event": "quiesce complete" } ], "locks": [ { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/ auth v22 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:20:25.443949+0000 1=1+0) n(v0 rc2024-04-15T15:20:25.542946+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={5017=pAsLsXsFs/pAsLsXsFs@8,5023=pAsLsXsFs/pAsLsXsFs@8,5280=pAsLsXs/-@9,5361=pAsLsXsFs/pAsLsXsFsxcral@30,5370=pAsLsXsFs/pAsLsXsFs@8} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x5580ff2c0b00]", "lock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "flags": 4, "wrlock_target": -1 }, { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/ auth v22 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:20:25.443949+0000 1=1+0) n(v0 rc2024-04-15T15:20:25.542946+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={5017=pAsLsXsFs/pAsLsXsFs@8,5023=pAsLsXsFs/pAsLsXsFs@8,5280=pAsLsXs/-@9,5361=pAsLsXsFs/pAsLsXsFsxcral@30,5370=pAsLsXsFs/pAsLsXsFs@8} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x5580ff2c0b00]", "lock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "flags": 2, "wrlock_target": -1 } ] } } 2024-04-15T15:20:28.110 ERROR:tasks.cephfs.test_quiesce:lock_type: iversion 2024-04-15T15:20:28.110 ERROR:tasks.cephfs.test_quiesce:cache: [ { "ino": 1099511629282, "rdev": 0, "ctime": "2024-04-15T15:20:25.542946+0000", "btime": "2024-04-15T15:20:25.443949+0000", "mode": 33188, "uid": 1000, "gid": 1301, "nlink": 1, "dir_layout": { "dir_hash": 0, "unused1": 0, "unused2": 0, "unused3": 0 }, "layout": { "stripe_unit": 4194304, "stripe_count": 1, "object_size": 4194304, "pool_id": 5, "pool_ns": "" }, "old_pools": [], "size": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "truncate_from": 0, "truncate_pending": 0, "mtime": "2024-04-15T15:20:25.457949+0000", "atime": "2024-04-15T15:20:25.457949+0000", "time_warp_seq": 2, "change_attr": 1, "export_pin": -1, "export_ephemeral_random_pin": 0, "export_ephemeral_distributed_pin": false, "quiesce_block": true, "client_ranges": [ { "client": 5361, "byte range": { "first": 0, "last": 4194304 }, "follows": 1 } ], "dirstat": { "version": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "change_attr": 0 }, "rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 0, "rsnaps": 0, "rctime": "2024-04-15T15:20:25.542946+0000" }, "accounted_rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 0, "rsnaps": 0, "rctime": "2024-04-15T15:20:25.542946+0000" }, "version": 6, "file_data_version": 0, "xattr_version": 1, "backtrace_version": 4, "stray_prior_path": "", "max_size_ever": 0, "quota": { "max_bytes": 0, "max_files": 0 }, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "symlink": "", "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ], "dirfragtree": { "splits": [] }, "old_inodes": [], "oldest_snap": 18446744073709551614, "damage_flags": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 1, "lock": 0, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 5, "versionlock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "authlock": { "gather_set": [], "state": "excl", "type": "iauth", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "linklock": {}, "dirfragtreelock": {}, "filelock": { "gather_set": [], "state": "excl", "type": "ifile", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "xattrlock": { "gather_set": [], "state": "excl", "type": "ixattr", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "snaplock": {}, "nestlock": { "gather_set": [], "state": "lock", "type": "inest", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "flocklock": {}, "policylock": {}, "quiescelock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "client_caps": [ { "client_id": 5361, "pending": "pAsxLsXsxFsxcrwb", "issued": "pAsxLsXsxFsxcrwb", "wanted": "pAsxXsxFxwb", "last_sent": 4 } ], "loner": 5361, "want_loner": 5361, "mds_caps_wanted": [], "dirfrags": [], "is_subvolume": false }, { "ino": 1099511627779, "rdev": 0, "ctime": "2024-04-15T15:20:25.443949+0000", "btime": "2024-04-15T15:20:12.433914+0000", "mode": 17407, "uid": 0, "gid": 0, "nlink": 1, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "layout": { "stripe_unit": 4194304, "stripe_count": 1, "object_size": 4194304, "pool_id": 5, "pool_ns": "" }, "old_pools": [], "size": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "truncate_from": 0, "truncate_pending": 0, "mtime": "2024-04-15T15:20:25.443949+0000", "atime": "2024-04-15T15:20:12.433914+0000", "time_warp_seq": 0, "change_attr": 3, "export_pin": -1, "export_ephemeral_random_pin": 0, "export_ephemeral_distributed_pin": false, "quiesce_block": false, "client_ranges": [], "dirstat": { "version": 0, "mtime": "2024-04-15T15:20:25.443949+0000", "num_files": 1, "num_subdirs": 0, "change_attr": 1 }, "rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:20:25.542946+0000" }, "accounted_rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:20:25.542946+0000" }, "version": 22, "file_data_version": 0, "xattr_version": 1, "backtrace_version": 2, "stray_prior_path": "", "max_size_ever": 0, "quota": { "max_bytes": 0, "max_files": 0 }, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "symlink": "", "xattrs": [], "dirfragtree": { "splits": [] }, "old_inodes": [], "oldest_snap": 18446744073709551614, "damage_flags": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8, "versionlock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "authlock": {}, "linklock": {}, "dirfragtreelock": {}, "filelock": {}, "xattrlock": {}, "snaplock": {}, "nestlock": { "gather_set": [], "state": "lock", "type": "inest", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "flocklock": {}, "policylock": {}, "quiescelock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "client_caps": [ { "client_id": 5017, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 }, { "client_id": 5023, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 }, { "client_id": 5280, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 9 }, { "client_id": 5361, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFsxcral", "last_sent": 30 }, { "client_id": 5370, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 } ], "loner": -1, "want_loner": -1, "mds_caps_wanted": [], "dirfrags": [ { "path": "/volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b", "dirfrag": "0x10000000003", "snapid_first": 2, "projected_version": "7", "version": "7", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "dentries": [ { "path": "volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/pyvenv.cfg", "path_ino": 1, "snap_first": 2, "snap_last": 18446744073709551614, "is_primary": false, "is_remote": false, "is_null": true, "is_new": false, "inode": 0, "remote_type": "", "version": 1, "projected_version": 0, "auth_pins": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "is_frozen": false, "is_freezing": false, "pins": { "lock": 0, "authpin": 0 }, "nref": 0, "lock": {}, "versionlock": { "gather_set": [], "state": "lock", "type": "dversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth" ] }, { "path": "volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/file", "path_ino": 1, "snap_first": 2, "snap_last": 18446744073709551614, "is_primary": true, "is_remote": false, "is_null": false, "is_new": false, "inode": 1099511629282, "remote_type": "", "version": 6, "projected_version": 6, "auth_pins": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "is_frozen": false, "is_freezing": false, "pins": { "request": 0, "lock": 0, "inodepin": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "lock": {}, "versionlock": { "gather_set": [], "state": "lock", "type": "dversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth", "dirty" ] } ] } ], "is_subvolume": false } ] 2024-04-15T15:20:28.111 ERROR:tasks.cephfs.test_quiesce:ops: { "1099511627779": { "description": "internal op quiesce_inode:mds.0:2 fp=#0x10000000003", "initiated_at": "2024-04-15T15:20:25.848020+0000", "age": 1.07623227, "duration": 1.076371371, "continuous": true, "type_data": { "result": -2147483648, "flag_point": "quiesce complete", "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 }, "op_type": "internal_op", "internal_op": 5385, "op_name": "quiesce_inode", "events": [ { "time": "2024-04-15T15:20:25.848020+0000", "event": "initiated" }, { "time": "2024-04-15T15:20:25.848020+0000", "event": "header_read" }, { "time": "2024-04-15T15:20:25.848020+0000", "event": "throttled" }, { "time": "2024-04-15T15:20:25.848020+0000", "event": "all_read" }, { "time": "2024-04-15T15:20:25.848020+0000", "event": "dispatched" }, { "time": "2024-04-15T15:20:25.848429+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:20:25.849067+0000", "event": "acquired locks" }, { "time": "2024-04-15T15:20:25.852708+0000", "event": "quiesce complete" } ], "locks": [ { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/ auth v22 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:20:25.443949+0000 1=1+0) n(v0 rc2024-04-15T15:20:25.542946+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={5017=pAsLsXsFs/pAsLsXsFs@8,5023=pAsLsXsFs/pAsLsXsFs@8,5280=pAsLsXs/-@9,5361=pAsLsXsFs/pAsLsXsFsxcral@30,5370=pAsLsXsFs/pAsLsXsFs@8} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x5580ff2c0b00]", "lock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "flags": 4, "wrlock_target": -1 }, { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/72ad24cb-8ea6-4d8a-90c9-5d14f4969d4b/ auth v22 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:20:25.443949+0000 1=1+0) n(v0 rc2024-04-15T15:20:25.542946+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={5017=pAsLsXsFs/pAsLsXsFs@8,5023=pAsLsXsFs/pAsLsXsFs@8,5280=pAsLsXs/-@9,5361=pAsLsXsFs/pAsLsXsFsxcral@30,5370=pAsLsXsFs/pAsLsXsFs@8} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x5580ff2c0b00]", "lock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "flags": 2, "wrlock_target": -1 } ] } }, "1099511629282": { "description": "internal op quiesce_inode:mds.0:3 fp=#0x100000005e2", "initiated_at": "2024-04-15T15:20:25.849283+0000", "age": 1.074969252, "duration": 1.075199891, "continuous": true, "type_data": { "result": -2147483648, "flag_point": "quiesce blocked", "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 3 }, "op_type": "internal_op", "internal_op": 5385, "op_name": "quiesce_inode", "events": [ { "time": "2024-04-15T15:20:25.849283+0000", "event": "initiated" }, { "time": "2024-04-15T15:20:25.849283+0000", "event": "header_read" }, { "time": "2024-04-15T15:20:25.849283+0000", "event": "throttled" }, { "time": "2024-04-15T15:20:25.849283+0000", "event": "all_read" }, { "time": "2024-04-15T15:20:25.849283+0000", "event": "dispatched" }, { "time": "2024-04-15T15:20:25.849403+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:20:25.850045+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:20:25.852362+0000", "event": "acquired locks" }, { "time": "2024-04-15T15:20:25.852364+0000", "event": "quiesce blocked" } ], "locks": [] } } } 2024-04-15T15:20:28.111 INFO:tasks.cephfs.test_quiesce:making archive directory /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/quiesce 2024-04-15T15:20:28.114 ERROR:tasks.cephfs.test_quiesce:cache written to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/quiesce/cacheeuz4cmnn 2024-04-15T15:20:28.115 ERROR:tasks.cephfs.test_quiesce:ops written to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/quiesce/opsegucv6h7 2024-04-15T15:20:28.116 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-15T15:20:28.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.349+0000 7fd01a600640 1 Processor -- start 2024-04-15T15:20:28.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.349+0000 7fd01a600640 1 -- start start 2024-04-15T15:20:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.351+0000 7fd01a600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd01415b410 0x7fd01415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.351+0000 7fd01a600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0140cf530 0x7fd0140cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.351+0000 7fd01a600640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd01415cae0 con 0x7fd01415f110 2024-04-15T15:20:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.351+0000 7fd01a600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd0140cd360 con 0x7fd01415b410 2024-04-15T15:20:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.351+0000 7fd01a600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd0140cd640 con 0x7fd0140cf530 2024-04-15T15:20:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.351+0000 7fd019c00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd01415f110 0x7fd01415f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42192/0 (socket says 172.21.15.49:42192) 2024-04-15T15:20:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.351+0000 7fd019c00640 1 -- 172.21.15.49:0/857299129 learned_addr learned my addr 172.21.15.49:0/857299129 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.351+0000 7fd019200640 1 --2- 172.21.15.49:0/857299129 >> v2:172.21.15.49:3300/0 conn(0x7fd01415b410 0x7fd01415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.352+0000 7fd013e00640 1 --2- 172.21.15.49:0/857299129 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0140cf530 0x7fd0140cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.352+0000 7fd019200640 1 -- 172.21.15.49:0/857299129 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0140cf530 msgr2=0x7fd0140cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.352+0000 7fd019200640 1 --2- 172.21.15.49:0/857299129 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0140cf530 0x7fd0140cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.352+0000 7fd013400640 1 -- 172.21.15.49:0/857299129 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2206135016 0 0) 0x7fd00c001490 con 0x7fd01415f110 2024-04-15T15:20:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.352+0000 7fd019200640 1 -- 172.21.15.49:0/857299129 >> v1:172.21.15.49:6789/0 conn(0x7fd01415f110 legacy=0x7fd01415f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.352+0000 7fd019200640 1 -- 172.21.15.49:0/857299129 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0140cd920 con 0x7fd01415b410 2024-04-15T15:20:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.353+0000 7fd019200640 1 --2- 172.21.15.49:0/857299129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415b410 0x7fd01415c5a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd0040097a0 tx=0x7fd00402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dbfb909155620e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.353+0000 7fd013400640 1 -- 172.21.15.49:0/857299129 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004031780 con 0x7fd01415b410 2024-04-15T15:20:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.353+0000 7fd013400640 1 -- 172.21.15.49:0/857299129 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd004037070 con 0x7fd01415b410 2024-04-15T15:20:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.353+0000 7fd013400640 1 -- 172.21.15.49:0/857299129 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004031b40 con 0x7fd01415b410 2024-04-15T15:20:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.354+0000 7fd01a600640 1 -- 172.21.15.49:0/857299129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415b410 msgr2=0x7fd01415c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.354+0000 7fd01a600640 1 --2- 172.21.15.49:0/857299129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415b410 0x7fd01415c5a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd0040097a0 tx=0x7fd00402b820 comp rx=0 tx=0).stop 2024-04-15T15:20:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.354+0000 7fd01a600640 1 -- 172.21.15.49:0/857299129 shutdown_connections 2024-04-15T15:20:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.354+0000 7fd01a600640 1 --2- 172.21.15.49:0/857299129 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0140cf530 0x7fd0140cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.354+0000 7fd01a600640 1 --2- 172.21.15.49:0/857299129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415b410 0x7fd01415c5a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.354+0000 7fd01a600640 1 -- 172.21.15.49:0/857299129 >> 172.21.15.49:0/857299129 conn(0x7fd0140b23a0 msgr2=0x7fd0140b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.354+0000 7fd01a600640 1 -- 172.21.15.49:0/857299129 shutdown_connections 2024-04-15T15:20:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.354+0000 7fd01a600640 1 -- 172.21.15.49:0/857299129 wait complete. 2024-04-15T15:20:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.356+0000 7fd01a600640 1 Processor -- start 2024-04-15T15:20:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.356+0000 7fd01a600640 1 -- start start 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.357+0000 7fd01a600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0140cf530 0x7fd01414df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.357+0000 7fd01a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd01415b410 0x7fd0140ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.357+0000 7fd01a600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415f110 0x7fd01415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.357+0000 7fd01a600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd0140cff00 con 0x7fd01415f110 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.357+0000 7fd01a600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd0140cdee0 con 0x7fd01415b410 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.357+0000 7fd01a600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd0140cd030 con 0x7fd0140cf530 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.358+0000 7fd019c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415f110 0x7fd01415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.358+0000 7fd019c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415f110 0x7fd01415b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43676/0 (socket says 172.21.15.49:43676) 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.358+0000 7fd019c00640 1 -- 172.21.15.49:0/1782327754 learned_addr learned my addr 172.21.15.49:0/1782327754 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.358+0000 7fd019200640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0140cf530 0x7fd01414df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.358+0000 7fd013e00640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd01415b410 0x7fd0140ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.358+0000 7fd019c00640 1 -- 172.21.15.49:0/1782327754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0140cf530 msgr2=0x7fd01414df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.358+0000 7fd019c00640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0140cf530 0x7fd01414df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.358+0000 7fd019c00640 1 -- 172.21.15.49:0/1782327754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd01415b410 msgr2=0x7fd0140ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.358+0000 7fd019c00640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd01415b410 0x7fd0140ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.358+0000 7fd019c00640 1 -- 172.21.15.49:0/1782327754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0140b4270 con 0x7fd01415f110 2024-04-15T15:20:28.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.359+0000 7fd019c00640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415f110 0x7fd01415b7f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00c730 tx=0x7fd00c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:28.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.359+0000 7fd010c00640 1 -- 172.21.15.49:0/1782327754 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c018070 con 0x7fd01415f110 2024-04-15T15:20:28.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.359+0000 7fd010c00640 1 -- 172.21.15.49:0/1782327754 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd00c00ce40 con 0x7fd01415f110 2024-04-15T15:20:28.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.359+0000 7fd010c00640 1 -- 172.21.15.49:0/1782327754 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c00a860 con 0x7fd01415f110 2024-04-15T15:20:28.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.359+0000 7fd01a600640 1 -- 172.21.15.49:0/1782327754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0140b4550 con 0x7fd01415f110 2024-04-15T15:20:28.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.359+0000 7fd01a600640 1 -- 172.21.15.49:0/1782327754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0140adc30 con 0x7fd01415f110 2024-04-15T15:20:28.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.360+0000 7fd01a600640 1 -- 172.21.15.49:0/1782327754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0140755c0 con 0x7fd01415f110 2024-04-15T15:20:28.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.361+0000 7fd010c00640 1 -- 172.21.15.49:0/1782327754 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd00c00aa00 con 0x7fd01415f110 2024-04-15T15:20:28.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.362+0000 7fd010c00640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcff006c980 0x7fcff006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.362+0000 7fd010c00640 1 -- 172.21.15.49:0/1782327754 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd00c08b210 con 0x7fd01415f110 2024-04-15T15:20:28.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.363+0000 7fd019200640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcff006c980 0x7fcff006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.363+0000 7fd019200640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcff006c980 0x7fcff006ee40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd004036000 tx=0x7fd00403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.367+0000 7fd010c00640 1 -- 172.21.15.49:0/1782327754 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd01415cae0 con 0x7fd01415f110 2024-04-15T15:20:28.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.616+0000 7fd01a600640 1 -- 172.21.15.49:0/1782327754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd01406fa40 con 0x7fd01415f110 2024-04-15T15:20:28.619 INFO:tasks.ceph.mon.a.smithi049.stderr:2024-04-15T15:20:28.617+0000 7fbf18400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-15T15:20:28.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.621+0000 7fd010c00640 1 -- 172.21.15.49:0/1782327754 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fd00c0595e0 con 0x7fd01415f110 2024-04-15T15:20:28.624 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-15T15:20:28.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.625+0000 7fd01a600640 1 -- 172.21.15.49:0/1782327754 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcff006c980 msgr2=0x7fcff006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.625+0000 7fd01a600640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcff006c980 0x7fcff006ee40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd004036000 tx=0x7fd00403d040 comp rx=0 tx=0).stop 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.625+0000 7fd01a600640 1 -- 172.21.15.49:0/1782327754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415f110 msgr2=0x7fd01415b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.625+0000 7fd01a600640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415f110 0x7fd01415b7f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00c730 tx=0x7fd00c00cbf0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.625+0000 7fd01a600640 1 -- 172.21.15.49:0/1782327754 shutdown_connections 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.625+0000 7fd01a600640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fcff006c980 0x7fcff006ee40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.625+0000 7fd01a600640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd01415f110 0x7fd01415b7f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.625+0000 7fd01a600640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd01415b410 0x7fd0140ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.625+0000 7fd01a600640 1 --2- 172.21.15.49:0/1782327754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0140cf530 0x7fd01414df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.625+0000 7fd01a600640 1 -- 172.21.15.49:0/1782327754 >> 172.21.15.49:0/1782327754 conn(0x7fd0140b23a0 msgr2=0x7fd0140b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.626+0000 7fd01a600640 1 -- 172.21.15.49:0/1782327754 shutdown_connections 2024-04-15T15:20:28.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.626+0000 7fd01a600640 1 -- 172.21.15.49:0/1782327754 wait complete. 2024-04-15T15:20:28.649 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-15T15:20:28.711 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2024-04-15T15:20:28.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.845+0000 7f7761600640 1 Processor -- start 2024-04-15T15:20:28.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.845+0000 7f7761600640 1 -- start start 2024-04-15T15:20:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.847+0000 7f7761600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c162f90 0x7f775c0ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.847+0000 7f7761600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f775c072860 0x7f775c166e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.847+0000 7f7761600640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f775c0cec10 con 0x7f775c0afbc0 2024-04-15T15:20:28.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.847+0000 7f7761600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f775c0b46c0 con 0x7f775c072860 2024-04-15T15:20:28.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.847+0000 7f7761600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f775c06fe90 con 0x7f775c162f90 2024-04-15T15:20:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.847+0000 7f7753e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f775c0afbc0 0x7f775c072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42214/0 (socket says 172.21.15.49:42214) 2024-04-15T15:20:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.847+0000 7f7753e00640 1 -- 172.21.15.49:0/3469513309 learned_addr learned my addr 172.21.15.49:0/3469513309 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775aa00640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c162f90 0x7f775c0ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775b400640 1 --2- 172.21.15.49:0/3469513309 >> v2:172.21.15.107:3300/0 conn(0x7f775c072860 0x7f775c166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837756078 0 0) 0x7f775c0cec10 con 0x7f775c0afbc0 2024-04-15T15:20:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7740003520 con 0x7f775c0afbc0 2024-04-15T15:20:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 593032845 0 0) 0x7f7740003520 con 0x7f775c0afbc0 2024-04-15T15:20:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c162f90 msgr2=0x7f775c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775a000640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c162f90 0x7f775c0ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 >> v2:172.21.15.107:3300/0 conn(0x7f775c072860 msgr2=0x7f775c166e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775a000640 1 --2- 172.21.15.49:0/3469513309 >> v2:172.21.15.107:3300/0 conn(0x7f775c072860 0x7f775c166e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c0b3ab0 con 0x7f775c0afbc0 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f7744001670 con 0x7f775c0afbc0 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775aa00640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c162f90 0x7f775c0ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.848+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 >> v1:172.21.15.49:6789/0 conn(0x7f775c0afbc0 legacy=0x7f775c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f775a000640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77400039b0 0x7f7740003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f775a000640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7740004800 0x7f7740004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f775a000640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c162f90 0x7f77400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f775c0b46c0 con 0x7f7740004800 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f775c06fe90 con 0x7f77400039b0 2024-04-15T15:20:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f774000b9c0 con 0x7f775c162f90 2024-04-15T15:20:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f7753e00640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7740004800 0x7f7740004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f775aa00640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77400039b0 0x7f7740003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f775b400640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c162f90 0x7f77400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f7753e00640 1 -- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c162f90 msgr2=0x7f77400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f7753e00640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c162f90 0x7f77400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f7753e00640 1 -- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77400039b0 msgr2=0x7f7740003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.849+0000 7f7753e00640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77400039b0 0x7f7740003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.850+0000 7f7753e00640 1 -- 172.21.15.49:0/3469513309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7740003520 con 0x7f7740004800 2024-04-15T15:20:28.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.850+0000 7f775aa00640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77400039b0 0x7f7740003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:28.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.851+0000 7f7753e00640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7740004800 0x7f7740004c80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f774400c240 tx=0x7f7740005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1c9938c0b101e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.851+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744007d30 con 0x7f7740004800 2024-04-15T15:20:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.851+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7744030070 con 0x7f7740004800 2024-04-15T15:20:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.851+0000 7f775a000640 1 -- 172.21.15.49:0/3469513309 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774402cd60 con 0x7f7740004800 2024-04-15T15:20:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.851+0000 7f7761600640 1 -- 172.21.15.49:0/3469513309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7740004800 msgr2=0x7f7740004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.851+0000 7f7761600640 1 --2- 172.21.15.49:0/3469513309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7740004800 0x7f7740004c80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f774400c240 tx=0x7f7740005830 comp rx=0 tx=0).stop 2024-04-15T15:20:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.852+0000 7f775b400640 1 -- 172.21.15.49:0/3469513309 reap_dead start 2024-04-15T15:20:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.852+0000 7f7761600640 1 -- 172.21.15.49:0/3469513309 shutdown_connections 2024-04-15T15:20:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.852+0000 7f7761600640 1 -- 172.21.15.49:0/3469513309 >> 172.21.15.49:0/3469513309 conn(0x7f775c0b2460 msgr2=0x7f775c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.852+0000 7f7761600640 1 -- 172.21.15.49:0/3469513309 shutdown_connections 2024-04-15T15:20:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.852+0000 7f7761600640 1 -- 172.21.15.49:0/3469513309 wait complete. 2024-04-15T15:20:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.854+0000 7f7761600640 1 Processor -- start 2024-04-15T15:20:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.854+0000 7f7761600640 1 -- start start 2024-04-15T15:20:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.855+0000 7f7761600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740004800 0x7f775c1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.855+0000 7f7761600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f775c15ab20 0x7f775c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.855+0000 7f7761600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c15d840 0x7f775c158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.855+0000 7f7761600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f775c0b4350 con 0x7f775c15ab20 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.855+0000 7f7761600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f775c0b4070 con 0x7f7740004800 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.855+0000 7f7761600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f775c163370 con 0x7f775c15d840 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.855+0000 7f7753e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f775c15ab20 0x7f775c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.855+0000 7f7753e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f775c15ab20 0x7f775c15d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43702/0 (socket says 172.21.15.49:43702) 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.855+0000 7f7753e00640 1 -- 172.21.15.49:0/593581736 learned_addr learned my addr 172.21.15.49:0/593581736 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.855+0000 7f775aa00640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740004800 0x7f775c1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7753e00640 1 -- 172.21.15.49:0/593581736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c15d840 msgr2=0x7f775c158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7753e00640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c15d840 0x7f775c158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7753e00640 1 -- 172.21.15.49:0/593581736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740004800 msgr2=0x7f775c1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7753e00640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740004800 0x7f775c1549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7753e00640 1 -- 172.21.15.49:0/593581736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c15afa0 con 0x7f775c15ab20 2024-04-15T15:20:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7753e00640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f775c15ab20 0x7f775c15d300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f774400c360 tx=0x7f775c0b3c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:28.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7752a00640 1 -- 172.21.15.49:0/593581736 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744001c70 con 0x7f775c15ab20 2024-04-15T15:20:28.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7752a00640 1 -- 172.21.15.49:0/593581736 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7744004bf0 con 0x7f775c15ab20 2024-04-15T15:20:28.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7752a00640 1 -- 172.21.15.49:0/593581736 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744007070 con 0x7f775c15ab20 2024-04-15T15:20:28.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7761600640 1 -- 172.21.15.49:0/593581736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c159a80 con 0x7f775c15ab20 2024-04-15T15:20:28.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.856+0000 7f7761600640 1 -- 172.21.15.49:0/593581736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c15a040 con 0x7f775c15ab20 2024-04-15T15:20:28.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.858+0000 7f7761600640 1 -- 172.21.15.49:0/593581736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775c068c80 con 0x7f775c15ab20 2024-04-15T15:20:28.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.859+0000 7f7752a00640 1 -- 172.21.15.49:0/593581736 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7744047020 con 0x7f775c15ab20 2024-04-15T15:20:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.860+0000 7f7752a00640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f773806c9b0 0x7f773806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.860+0000 7f7752a00640 1 -- 172.21.15.49:0/593581736 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f77440ace10 con 0x7f775c15ab20 2024-04-15T15:20:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.860+0000 7f775aa00640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f773806c9b0 0x7f773806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.861+0000 7f775aa00640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f773806c9b0 0x7f773806ee70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f775c15c130 tx=0x7f774c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:28.864+0000 7f7752a00640 1 -- 172.21.15.49:0/593581736 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77440b0030 con 0x7f775c15ab20 2024-04-15T15:20:29.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.107+0000 7f7761600640 1 -- 172.21.15.49:0/593581736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f775c06f8d0 con 0x7f775c15ab20 2024-04-15T15:20:29.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.624+0000 7f7752a00640 1 -- 172.21.15.49:0/593581736 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v19) ==== 162+0+0 (secure 0 0 0) 0x7f774407b180 con 0x7f775c15ab20 2024-04-15T15:20:29.628 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-15T15:20:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 -- 172.21.15.49:0/593581736 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f773806c9b0 msgr2=0x7f773806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f773806c9b0 0x7f773806ee70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f775c15c130 tx=0x7f774c008040 comp rx=0 tx=0).stop 2024-04-15T15:20:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 -- 172.21.15.49:0/593581736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f775c15ab20 msgr2=0x7f775c15d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f775c15ab20 0x7f775c15d300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f774400c360 tx=0x7f775c0b3c80 comp rx=0 tx=0).stop 2024-04-15T15:20:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 -- 172.21.15.49:0/593581736 shutdown_connections 2024-04-15T15:20:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f773806c9b0 0x7f773806ee70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f775c15d840 0x7f775c158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f775c15ab20 0x7f775c15d300 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 --2- 172.21.15.49:0/593581736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740004800 0x7f775c1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 -- 172.21.15.49:0/593581736 >> 172.21.15.49:0/593581736 conn(0x7f775c0b2460 msgr2=0x7f775c074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:29.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.629+0000 7f7761600640 1 -- 172.21.15.49:0/593581736 shutdown_connections 2024-04-15T15:20:29.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.630+0000 7f7761600640 1 -- 172.21.15.49:0/593581736 wait complete. 2024-04-15T15:20:29.657 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:29.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.849+0000 7f2775e00640 1 Processor -- start 2024-04-15T15:20:29.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.850+0000 7f2775e00640 1 -- start start 2024-04-15T15:20:29.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.851+0000 7f2775e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27700cee00 0x7f277006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.851+0000 7f2775e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2770071f50 0x7f27700723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.851+0000 7f2775e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f277015dbb0 con 0x7f277015cbe0 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.851+0000 7f2775e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f277006f4f0 con 0x7f2770071f50 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.851+0000 7f2775e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f27700728f0 con 0x7f27700cee00 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.852+0000 7f276fe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2770071f50 0x7f27700723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.852+0000 7f276fe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2770071f50 0x7f27700723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43704/0 (socket says 172.21.15.49:43704) 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.852+0000 7f276fe00640 1 -- 172.21.15.49:0/1669460264 learned_addr learned my addr 172.21.15.49:0/1669460264 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.852+0000 7f2774a00640 1 --2- 172.21.15.49:0/1669460264 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27700cee00 0x7f277006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.852+0000 7f276fe00640 1 -- 172.21.15.49:0/1669460264 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27700cee00 msgr2=0x7f277006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.852+0000 7f276fe00640 1 --2- 172.21.15.49:0/1669460264 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27700cee00 0x7f277006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.852+0000 7f276fe00640 1 -- 172.21.15.49:0/1669460264 >> v1:172.21.15.107:6789/0 conn(0x7f277015cbe0 legacy=0x7f277015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.852+0000 7f276fe00640 1 -- 172.21.15.49:0/1669460264 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277015d8d0 con 0x7f2770071f50 2024-04-15T15:20:29.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.853+0000 7f276fe00640 1 --2- 172.21.15.49:0/1669460264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2770071f50 0x7f27700723b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f275c0098a0 tx=0x7f275c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cca6553a474649 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.854+0000 7f276f400640 1 -- 172.21.15.49:0/1669460264 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c035020 con 0x7f2770071f50 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.854+0000 7f276f400640 1 -- 172.21.15.49:0/1669460264 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c031640 con 0x7f2770071f50 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.854+0000 7f276f400640 1 -- 172.21.15.49:0/1669460264 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c02d040 con 0x7f2770071f50 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.854+0000 7f2775e00640 1 -- 172.21.15.49:0/1669460264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2770071f50 msgr2=0x7f27700723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.854+0000 7f2775e00640 1 --2- 172.21.15.49:0/1669460264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2770071f50 0x7f27700723b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f275c0098a0 tx=0x7f275c02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.855+0000 7f2775e00640 1 -- 172.21.15.49:0/1669460264 shutdown_connections 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.855+0000 7f2775e00640 1 --2- 172.21.15.49:0/1669460264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2770071f50 0x7f27700723b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.855+0000 7f2775e00640 1 --2- 172.21.15.49:0/1669460264 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27700cee00 0x7f277006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.855+0000 7f2775e00640 1 -- 172.21.15.49:0/1669460264 >> 172.21.15.49:0/1669460264 conn(0x7f277006b930 msgr2=0x7f277006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.855+0000 7f2775e00640 1 -- 172.21.15.49:0/1669460264 shutdown_connections 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.855+0000 7f2775e00640 1 -- 172.21.15.49:0/1669460264 wait complete. 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.856+0000 7f2775e00640 1 Processor -- start 2024-04-15T15:20:29.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.856+0000 7f2775e00640 1 -- start start 2024-04-15T15:20:29.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.857+0000 7f2775e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2770071f50 0x7f2770143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.857+0000 7f2775e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27700cee00 0x7f277015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.857+0000 7f2775e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f277015cbe0 0x7f277015f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.857+0000 7f2775e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f277006e3a0 con 0x7f27700cee00 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.857+0000 7f2775e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f277006e0c0 con 0x7f2770071f50 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.857+0000 7f2775e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f277015dda0 con 0x7f277015cbe0 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f276fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27700cee00 0x7f277015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f276fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27700cee00 0x7f277015f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43720/0 (socket says 172.21.15.49:43720) 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f276fe00640 1 -- 172.21.15.49:0/2443607252 learned_addr learned my addr 172.21.15.49:0/2443607252 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f2775400640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f277015cbe0 0x7f277015f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f2774a00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2770071f50 0x7f2770143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f276fe00640 1 -- 172.21.15.49:0/2443607252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f277015cbe0 msgr2=0x7f277015f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f276fe00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f277015cbe0 0x7f277015f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f276fe00640 1 -- 172.21.15.49:0/2443607252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2770071f50 msgr2=0x7f2770143ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f276fe00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2770071f50 0x7f2770143ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f276fe00640 1 -- 172.21.15.49:0/2443607252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2770144410 con 0x7f27700cee00 2024-04-15T15:20:29.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f2775400640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f277015cbe0 0x7f277015f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:29.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f2774a00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2770071f50 0x7f2770143ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:29.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.858+0000 7f276fe00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27700cee00 0x7f277015f1b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f275c02b930 tx=0x7f275c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:29.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.859+0000 7f276cc00640 1 -- 172.21.15.49:0/2443607252 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c040070 con 0x7f27700cee00 2024-04-15T15:20:29.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.859+0000 7f276cc00640 1 -- 172.21.15.49:0/2443607252 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c0399f0 con 0x7f27700cee00 2024-04-15T15:20:29.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.859+0000 7f2775e00640 1 -- 172.21.15.49:0/2443607252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27700aead0 con 0x7f27700cee00 2024-04-15T15:20:29.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.859+0000 7f276cc00640 1 -- 172.21.15.49:0/2443607252 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c035040 con 0x7f27700cee00 2024-04-15T15:20:29.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.859+0000 7f2775e00640 1 -- 172.21.15.49:0/2443607252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27700b3c40 con 0x7f27700cee00 2024-04-15T15:20:29.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.860+0000 7f2775e00640 1 -- 172.21.15.49:0/2443607252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27700b3780 con 0x7f27700cee00 2024-04-15T15:20:29.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.861+0000 7f276cc00640 1 -- 172.21.15.49:0/2443607252 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f275c04b060 con 0x7f27700cee00 2024-04-15T15:20:29.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.862+0000 7f276cc00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f275006c9b0 0x7f275006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:29.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.862+0000 7f276cc00640 1 -- 172.21.15.49:0/2443607252 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f275c0ad600 con 0x7f27700cee00 2024-04-15T15:20:29.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.863+0000 7f2774a00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f275006c9b0 0x7f275006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:29.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.863+0000 7f2774a00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f275006c9b0 0x7f275006ee70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2764006fd0 tx=0x7f2764008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:29.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:29.867+0000 7f276cc00640 1 -- 172.21.15.49:0/2443607252 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27700b3780 con 0x7f27700cee00 2024-04-15T15:20:30.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.112+0000 7f2775e00640 1 -- 172.21.15.49:0/2443607252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2770156eb0 con 0x7f27700cee00 2024-04-15T15:20:30.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.113+0000 7f276cc00640 1 -- 172.21.15.49:0/2443607252 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5181 (secure 0 0 0) 0x7f275c07b970 con 0x7f27700cee00 2024-04-15T15:20:30.116 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:30.116 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:00.037472+0000","modified":"2024-04-15T15:20:29.629927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"c","rank":0,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.49:6837/3168788676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":3168788676},{"type":"v1","addr":"172.21.15.49:6837","nonce":3168788676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-15T15:20:30.117 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 20 2024-04-15T15:20:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 -- 172.21.15.49:0/2443607252 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f275006c9b0 msgr2=0x7f275006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f275006c9b0 0x7f275006ee70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2764006fd0 tx=0x7f2764008040 comp rx=0 tx=0).stop 2024-04-15T15:20:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 -- 172.21.15.49:0/2443607252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27700cee00 msgr2=0x7f277015f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27700cee00 0x7f277015f1b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f275c02b930 tx=0x7f275c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 -- 172.21.15.49:0/2443607252 shutdown_connections 2024-04-15T15:20:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f275006c9b0 0x7f275006ee70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f277015cbe0 0x7f277015f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27700cee00 0x7f277015f1b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 --2- 172.21.15.49:0/2443607252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2770071f50 0x7f2770143ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 -- 172.21.15.49:0/2443607252 >> 172.21.15.49:0/2443607252 conn(0x7f277006b930 msgr2=0x7f27701555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:30.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 -- 172.21.15.49:0/2443607252 shutdown_connections 2024-04-15T15:20:30.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:30.116+0000 7f2775e00640 1 -- 172.21.15.49:0/2443607252 wait complete. 2024-04-15T15:20:30.541 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:20:00.037472+0000', 'modified': '2024-04-15T15:20:29.629927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5445': {'gid': 5445, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.49:6837/3168788676', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 3168788676}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 3168788676}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-15T15:20:30.541 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5445:up:replay 2024-04-15T15:20:31.542 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.768+0000 7f423f200640 1 Processor -- start 2024-04-15T15:20:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.768+0000 7f423f200640 1 -- start start 2024-04-15T15:20:31.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.769+0000 7f423f200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f42400d1c90 0x7f4240162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.769+0000 7f423f200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f42400cf110 0x7f42400d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.769+0000 7f423f200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42400d0090 con 0x7f4240162020 2024-04-15T15:20:31.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.769+0000 7f423f200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f4240163440 con 0x7f42400cf110 2024-04-15T15:20:31.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.769+0000 7f423f200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f42400b7b30 con 0x7f42400d1c90 2024-04-15T15:20:31.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.770+0000 7f423e800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f42400cf110 0x7f42400d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.770+0000 7f423de00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4240162020 0x7f42400d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42226/0 (socket says 172.21.15.49:42226) 2024-04-15T15:20:31.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.770+0000 7f423de00640 1 -- 172.21.15.49:0/1193995068 learned_addr learned my addr 172.21.15.49:0/1193995068 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:31.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.770+0000 7f423d400640 1 --2- 172.21.15.49:0/1193995068 >> v2:172.21.15.107:3300/0 conn(0x7f42400d1c90 0x7f4240162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.770+0000 7f423e800640 1 -- 172.21.15.49:0/1193995068 >> v2:172.21.15.107:3300/0 conn(0x7f42400d1c90 msgr2=0x7f4240162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:31.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.770+0000 7f423e800640 1 --2- 172.21.15.49:0/1193995068 >> v2:172.21.15.107:3300/0 conn(0x7f42400d1c90 0x7f4240162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.770+0000 7f423e800640 1 -- 172.21.15.49:0/1193995068 >> v1:172.21.15.49:6789/0 conn(0x7f4240162020 legacy=0x7f42400d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:31.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.770+0000 7f423e800640 1 -- 172.21.15.49:0/1193995068 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42400cfad0 con 0x7f42400cf110 2024-04-15T15:20:31.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.772+0000 7f423e800640 1 --2- 172.21.15.49:0/1193995068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 0x7f42400d3390 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f424008cb30 tx=0x7f423802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fa72f3c53b8fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:31.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.772+0000 7f423ca00640 1 -- 172.21.15.49:0/1193995068 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238039070 con 0x7f42400cf110 2024-04-15T15:20:31.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.773+0000 7f423ca00640 1 -- 172.21.15.49:0/1193995068 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f423802cc70 con 0x7f42400cf110 2024-04-15T15:20:31.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.773+0000 7f423ca00640 1 -- 172.21.15.49:0/1193995068 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238034660 con 0x7f42400cf110 2024-04-15T15:20:31.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.773+0000 7f423f200640 1 -- 172.21.15.49:0/1193995068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 msgr2=0x7f42400d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:31.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.773+0000 7f423f200640 1 --2- 172.21.15.49:0/1193995068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 0x7f42400d3390 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f424008cb30 tx=0x7f423802bec0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.773+0000 7f423f200640 1 -- 172.21.15.49:0/1193995068 shutdown_connections 2024-04-15T15:20:31.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.773+0000 7f423f200640 1 --2- 172.21.15.49:0/1193995068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 0x7f42400d3390 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.773+0000 7f423f200640 1 --2- 172.21.15.49:0/1193995068 >> v2:172.21.15.107:3300/0 conn(0x7f42400d1c90 0x7f4240162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.773+0000 7f423f200640 1 -- 172.21.15.49:0/1193995068 >> 172.21.15.49:0/1193995068 conn(0x7f424006cf00 msgr2=0x7f424006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:31.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.773+0000 7f423f200640 1 -- 172.21.15.49:0/1193995068 shutdown_connections 2024-04-15T15:20:31.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.774+0000 7f423f200640 1 -- 172.21.15.49:0/1193995068 wait complete. 2024-04-15T15:20:31.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423f200640 1 Processor -- start 2024-04-15T15:20:31.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423f200640 1 -- start start 2024-04-15T15:20:31.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 0x7f424015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42400d1c90 0x7f4240159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4240162020 0x7f42401f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42400d2b90 con 0x7f42400cf110 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42400cf880 con 0x7f4240162020 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f42400d2e70 con 0x7f42400d1c90 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 0x7f424015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 0x7f424015b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43732/0 (socket says 172.21.15.49:43732) 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.777+0000 7f423de00640 1 -- 172.21.15.49:0/1429175511 learned_addr learned my addr 172.21.15.49:0/1429175511 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.779+0000 7f423e800640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4240162020 0x7f42401f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.779+0000 7f423d400640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42400d1c90 0x7f4240159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.779+0000 7f423de00640 1 -- 172.21.15.49:0/1429175511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42400d1c90 msgr2=0x7f4240159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.779+0000 7f423de00640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42400d1c90 0x7f4240159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.779+0000 7f423de00640 1 -- 172.21.15.49:0/1429175511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4240162020 msgr2=0x7f42401f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:31.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.779+0000 7f423de00640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4240162020 0x7f42401f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:31.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.779+0000 7f423de00640 1 -- 172.21.15.49:0/1429175511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4240159060 con 0x7f42400cf110 2024-04-15T15:20:31.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.779+0000 7f423e800640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4240162020 0x7f42401f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:31.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.779+0000 7f423de00640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 0x7f424015b3f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f423000c400 tx=0x7f423000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:31.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.780+0000 7f4236000640 1 -- 172.21.15.49:0/1429175511 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230017070 con 0x7f42400cf110 2024-04-15T15:20:31.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.780+0000 7f4236000640 1 -- 172.21.15.49:0/1429175511 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4230005ce0 con 0x7f42400cf110 2024-04-15T15:20:31.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.780+0000 7f4236000640 1 -- 172.21.15.49:0/1429175511 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42300072f0 con 0x7f42400cf110 2024-04-15T15:20:31.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.780+0000 7f423f200640 1 -- 172.21.15.49:0/1429175511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42400cd510 con 0x7f42400cf110 2024-04-15T15:20:31.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.780+0000 7f423f200640 1 -- 172.21.15.49:0/1429175511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42400cdad0 con 0x7f42400cf110 2024-04-15T15:20:31.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.781+0000 7f423f200640 1 -- 172.21.15.49:0/1429175511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42400755c0 con 0x7f42400cf110 2024-04-15T15:20:31.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.782+0000 7f4236000640 1 -- 172.21.15.49:0/1429175511 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f423001a070 con 0x7f42400cf110 2024-04-15T15:20:31.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.783+0000 7f4236000640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f421c06c980 0x7f421c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:31.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.783+0000 7f4236000640 1 -- 172.21.15.49:0/1429175511 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f423008b070 con 0x7f42400cf110 2024-04-15T15:20:31.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.783+0000 7f423d400640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f421c06c980 0x7f421c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:31.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.784+0000 7f423d400640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f421c06c980 0x7f421c06ee40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4228006fd0 tx=0x7f4228008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:31.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:31.789+0000 7f4236000640 1 -- 172.21.15.49:0/1429175511 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42400d0090 con 0x7f42400cf110 2024-04-15T15:20:32.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.040+0000 7f423f200640 1 -- 172.21.15.49:0/1429175511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42400b7650 con 0x7f42400cf110 2024-04-15T15:20:32.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.041+0000 7f4236000640 1 -- 172.21.15.49:0/1429175511 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5181 (secure 0 0 0) 0x7f42300593e0 con 0x7f42400cf110 2024-04-15T15:20:32.043 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:32.043 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:00.037472+0000","modified":"2024-04-15T15:20:30.658327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"c","rank":0,"incarnation":20,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.49:6837/3168788676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":3168788676},{"type":"v1","addr":"172.21.15.49:6837","nonce":3168788676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-15T15:20:32.043 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 22 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 -- 172.21.15.49:0/1429175511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f421c06c980 msgr2=0x7f421c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f421c06c980 0x7f421c06ee40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4228006fd0 tx=0x7f4228008040 comp rx=0 tx=0).stop 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 -- 172.21.15.49:0/1429175511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 msgr2=0x7f424015b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 0x7f424015b3f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f423000c400 tx=0x7f423000c8c0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 -- 172.21.15.49:0/1429175511 shutdown_connections 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f421c06c980 0x7f421c06ee40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42400d1c90 0x7f4240159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4240162020 0x7f42401f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 --2- 172.21.15.49:0/1429175511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42400cf110 0x7f424015b3f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 -- 172.21.15.49:0/1429175511 >> 172.21.15.49:0/1429175511 conn(0x7f424006cf00 msgr2=0x7f424015eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:32.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 -- 172.21.15.49:0/1429175511 shutdown_connections 2024-04-15T15:20:32.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:32.044+0000 7f423f200640 1 -- 172.21.15.49:0/1429175511 wait complete. 2024-04-15T15:20:32.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:20:00.037472+0000', 'modified': '2024-04-15T15:20:30.658327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5445': {'gid': 5445, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.49:6837/3168788676', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 3168788676}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 3168788676}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-15T15:20:32.468 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5445:up:rejoin 2024-04-15T15:20:33.470 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:33.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.707+0000 7efc63200640 1 Processor -- start 2024-04-15T15:20:33.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.708+0000 7efc63200640 1 -- start start 2024-04-15T15:20:33.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.709+0000 7efc63200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc64162020 0x7efc64162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.709+0000 7efc63200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efc640cf110 0x7efc640d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.709+0000 7efc63200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc640d0090 con 0x7efc640d1ac0 2024-04-15T15:20:33.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.709+0000 7efc63200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7efc640d1f50 con 0x7efc64162020 2024-04-15T15:20:33.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.709+0000 7efc63200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7efc64163440 con 0x7efc640cf110 2024-04-15T15:20:33.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.709+0000 7efc61e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc64162020 0x7efc64162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.709+0000 7efc61e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc64162020 0x7efc64162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43734/0 (socket says 172.21.15.49:43734) 2024-04-15T15:20:33.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.709+0000 7efc61e00640 1 -- 172.21.15.49:0/3444127059 learned_addr learned my addr 172.21.15.49:0/3444127059 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:33.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.710+0000 7efc61e00640 1 -- 172.21.15.49:0/3444127059 >> v2:172.21.15.107:3300/0 conn(0x7efc640cf110 msgr2=0x7efc640d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:33.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.710+0000 7efc61e00640 1 --2- 172.21.15.49:0/3444127059 >> v2:172.21.15.107:3300/0 conn(0x7efc640cf110 0x7efc640d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.710+0000 7efc61e00640 1 -- 172.21.15.49:0/3444127059 >> v1:172.21.15.107:6789/0 conn(0x7efc640d1ac0 legacy=0x7efc640b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:33.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.710+0000 7efc61e00640 1 -- 172.21.15.49:0/3444127059 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc640cfad0 con 0x7efc64162020 2024-04-15T15:20:33.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.711+0000 7efc61e00640 1 --2- 172.21.15.49:0/3444127059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc64162020 0x7efc64162f00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efc6408cb30 tx=0x7efc4c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413ed2ad01f8b688 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.711+0000 7efc60a00640 1 -- 172.21.15.49:0/3444127059 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c035020 con 0x7efc64162020 2024-04-15T15:20:33.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.711+0000 7efc60a00640 1 -- 172.21.15.49:0/3444127059 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc4c031870 con 0x7efc64162020 2024-04-15T15:20:33.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.711+0000 7efc60a00640 1 -- 172.21.15.49:0/3444127059 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c031c40 con 0x7efc64162020 2024-04-15T15:20:33.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.712+0000 7efc63200640 1 -- 172.21.15.49:0/3444127059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc64162020 msgr2=0x7efc64162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.712+0000 7efc63200640 1 --2- 172.21.15.49:0/3444127059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc64162020 0x7efc64162f00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efc6408cb30 tx=0x7efc4c02b910 comp rx=0 tx=0).stop 2024-04-15T15:20:33.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.712+0000 7efc63200640 1 -- 172.21.15.49:0/3444127059 shutdown_connections 2024-04-15T15:20:33.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.712+0000 7efc63200640 1 --2- 172.21.15.49:0/3444127059 >> v2:172.21.15.107:3300/0 conn(0x7efc640cf110 0x7efc640d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.712+0000 7efc63200640 1 --2- 172.21.15.49:0/3444127059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc64162020 0x7efc64162f00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.712+0000 7efc63200640 1 -- 172.21.15.49:0/3444127059 >> 172.21.15.49:0/3444127059 conn(0x7efc6406cf00 msgr2=0x7efc6406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:33.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.712+0000 7efc63200640 1 -- 172.21.15.49:0/3444127059 shutdown_connections 2024-04-15T15:20:33.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.712+0000 7efc63200640 1 -- 172.21.15.49:0/3444127059 wait complete. 2024-04-15T15:20:33.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.714+0000 7efc63200640 1 Processor -- start 2024-04-15T15:20:33.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.714+0000 7efc63200640 1 -- start start 2024-04-15T15:20:33.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.715+0000 7efc63200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc640cf110 0x7efc641f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.715+0000 7efc63200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640d1ac0 0x7efc640c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.715+0000 7efc63200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc64162020 0x7efc6414e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.715+0000 7efc63200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efc640cf880 con 0x7efc640cf110 2024-04-15T15:20:33.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.715+0000 7efc63200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc640b7b30 con 0x7efc640d1ac0 2024-04-15T15:20:33.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.715+0000 7efc63200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7efc640d2b90 con 0x7efc64162020 2024-04-15T15:20:33.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.715+0000 7efc61e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc640cf110 0x7efc641f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.715+0000 7efc61e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc640cf110 0x7efc641f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43740/0 (socket says 172.21.15.49:43740) 2024-04-15T15:20:33.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.715+0000 7efc61e00640 1 -- 172.21.15.49:0/2701086154 learned_addr learned my addr 172.21.15.49:0/2701086154 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.716+0000 7efc62800640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc64162020 0x7efc6414e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.716+0000 7efc61e00640 1 -- 172.21.15.49:0/2701086154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc64162020 msgr2=0x7efc6414e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.716+0000 7efc61400640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640d1ac0 0x7efc640c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.716+0000 7efc61e00640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc64162020 0x7efc6414e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.716+0000 7efc61e00640 1 -- 172.21.15.49:0/2701086154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640d1ac0 msgr2=0x7efc640c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.716+0000 7efc61e00640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640d1ac0 0x7efc640c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.716+0000 7efc61e00640 1 -- 172.21.15.49:0/2701086154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6414ed50 con 0x7efc640cf110 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.716+0000 7efc62800640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc64162020 0x7efc6414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.716+0000 7efc61e00640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc640cf110 0x7efc641f8eb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efc4c000c00 tx=0x7efc4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.717+0000 7efc5a000640 1 -- 172.21.15.49:0/2701086154 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c03ad60 con 0x7efc640cf110 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.717+0000 7efc5a000640 1 -- 172.21.15.49:0/2701086154 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc4c0443f0 con 0x7efc640cf110 2024-04-15T15:20:33.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.717+0000 7efc5a000640 1 -- 172.21.15.49:0/2701086154 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c0447e0 con 0x7efc640cf110 2024-04-15T15:20:33.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.717+0000 7efc63200640 1 -- 172.21.15.49:0/2701086154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc641e8e80 con 0x7efc640cf110 2024-04-15T15:20:33.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.717+0000 7efc63200640 1 -- 172.21.15.49:0/2701086154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc641e9440 con 0x7efc640cf110 2024-04-15T15:20:33.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.718+0000 7efc63200640 1 -- 172.21.15.49:0/2701086154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc640755c0 con 0x7efc640cf110 2024-04-15T15:20:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.719+0000 7efc5a000640 1 -- 172.21.15.49:0/2701086154 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7efc4c038020 con 0x7efc640cf110 2024-04-15T15:20:33.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.720+0000 7efc5a000640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc4006c980 0x7efc4006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:33.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.720+0000 7efc5a000640 1 -- 172.21.15.49:0/2701086154 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7efc4c0ae0d0 con 0x7efc640cf110 2024-04-15T15:20:33.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.721+0000 7efc61400640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc4006c980 0x7efc4006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:33.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.721+0000 7efc61400640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc4006c980 0x7efc4006ee40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efc641613b0 tx=0x7efc540072f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:33.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.725+0000 7efc5a000640 1 -- 172.21.15.49:0/2701086154 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc4c03f070 con 0x7efc640cf110 2024-04-15T15:20:33.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.971+0000 7efc63200640 1 -- 172.21.15.49:0/2701086154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc640b7650 con 0x7efc640cf110 2024-04-15T15:20:33.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.973+0000 7efc5a000640 1 -- 172.21.15.49:0/2701086154 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5188 (secure 0 0 0) 0x7efc640d0090 con 0x7efc640cf110 2024-04-15T15:20:33.974 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:33.975 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:00.037472+0000","modified":"2024-04-15T15:20:32.648230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/3168788676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":3168788676},{"type":"v1","addr":"172.21.15.49:6837","nonce":3168788676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5445,"qdb_cluster":[5445]},"id":2}]} 2024-04-15T15:20:33.975 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-04-15T15:20:33.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.975+0000 7efc63200640 1 -- 172.21.15.49:0/2701086154 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc4006c980 msgr2=0x7efc4006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.975+0000 7efc63200640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc4006c980 0x7efc4006ee40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efc641613b0 tx=0x7efc540072f0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.975+0000 7efc63200640 1 -- 172.21.15.49:0/2701086154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc640cf110 msgr2=0x7efc641f8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:33.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.975+0000 7efc63200640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc640cf110 0x7efc641f8eb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efc4c000c00 tx=0x7efc4c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.975+0000 7efc63200640 1 -- 172.21.15.49:0/2701086154 shutdown_connections 2024-04-15T15:20:33.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.975+0000 7efc63200640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc4006c980 0x7efc4006ee40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.975+0000 7efc63200640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc64162020 0x7efc6414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.975+0000 7efc63200640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640d1ac0 0x7efc640c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.975+0000 7efc63200640 1 --2- 172.21.15.49:0/2701086154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc640cf110 0x7efc641f8eb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:33.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.975+0000 7efc63200640 1 -- 172.21.15.49:0/2701086154 >> 172.21.15.49:0/2701086154 conn(0x7efc6406cf00 msgr2=0x7efc640cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:33.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.976+0000 7efc63200640 1 -- 172.21.15.49:0/2701086154 shutdown_connections 2024-04-15T15:20:33.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:33.976+0000 7efc63200640 1 -- 172.21.15.49:0/2701086154 wait complete. 2024-04-15T15:20:34.399 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:20:00.037472+0000', 'modified': '2024-04-15T15:20:32.648230+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5445': {'gid': 5445, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6837/3168788676', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 3168788676}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 3168788676}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5445, 'qdb_cluster': [5445]} 2024-04-15T15:20:34.399 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-15T15:20:34.399 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-15T15:20:34.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.631+0000 7fc637200640 1 Processor -- start 2024-04-15T15:20:34.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.631+0000 7fc637200640 1 -- start start 2024-04-15T15:20:34.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.633+0000 7fc637200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc6380b59b0 0x7fc6380b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.633+0000 7fc637200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6380cd940 con 0x7fc63815d770 2024-04-15T15:20:34.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.633+0000 7fc637200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6380cdc20 con 0x7fc6380b0cb0 2024-04-15T15:20:34.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.633+0000 7fc637200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fc6380ce550 con 0x7fc6380b59b0 2024-04-15T15:20:34.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.633+0000 7fc635400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc63815d770 0x7fc6380cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60088/0 (socket says 172.21.15.49:60088) 2024-04-15T15:20:34.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.633+0000 7fc635400640 1 -- 172.21.15.49:0/3020495260 learned_addr learned my addr 172.21.15.49:0/3020495260 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:34.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.634+0000 7fc636800640 1 --2- 172.21.15.49:0/3020495260 >> v2:172.21.15.107:3300/0 conn(0x7fc6380b59b0 0x7fc6380b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.634+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3966473560 0 0) 0x7fc6380cd940 con 0x7fc63815d770 2024-04-15T15:20:34.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.634+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc618003520 con 0x7fc63815d770 2024-04-15T15:20:34.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.634+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459967027 0 0) 0x7fc6380cdc20 con 0x7fc6380b0cb0 2024-04-15T15:20:34.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.634+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6380cd940 con 0x7fc6380b0cb0 2024-04-15T15:20:34.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.634+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1499401045 0 0) 0x7fc618003520 con 0x7fc63815d770 2024-04-15T15:20:34.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 >> v2:172.21.15.107:3300/0 conn(0x7fc6380b59b0 msgr2=0x7fc6380b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 --2- 172.21.15.49:0/3020495260 >> v2:172.21.15.107:3300/0 conn(0x7fc6380b59b0 0x7fc6380b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 >> v1:172.21.15.107:6789/0 conn(0x7fc6380b0cb0 legacy=0x7fc6380ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6380cdf00 con 0x7fc63815d770 2024-04-15T15:20:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fc6200019f0 con 0x7fc63815d770 2024-04-15T15:20:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 >> v1:172.21.15.49:6789/0 conn(0x7fc63815d770 legacy=0x7fc6380cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc618003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc618004800 0x7fc618004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180051c0 0x7fc61800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6380ce550 con 0x7fc618003b90 2024-04-15T15:20:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6180104a0 con 0x7fc6180051c0 2024-04-15T15:20:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc6180106a0 con 0x7fc618004800 2024-04-15T15:20:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.635+0000 7fc635e00640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc618003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.636+0000 7fc636800640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180051c0 0x7fc61800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.636+0000 7fc635400640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc618004800 0x7fc618004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.636+0000 7fc635e00640 1 -- 172.21.15.49:0/3020495260 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc618004800 msgr2=0x7fc618004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.636+0000 7fc635e00640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc618004800 0x7fc618004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.636+0000 7fc635e00640 1 -- 172.21.15.49:0/3020495260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180051c0 msgr2=0x7fc61800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.636+0000 7fc635e00640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180051c0 0x7fc61800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.636+0000 7fc636800640 1 -- 172.21.15.49:0/3020495260 reap_dead start 2024-04-15T15:20:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.636+0000 7fc635e00640 1 -- 172.21.15.49:0/3020495260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618003520 con 0x7fc618003b90 2024-04-15T15:20:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.636+0000 7fc635400640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc618004800 0x7fc618004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:34.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.637+0000 7fc635e00640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc618003f90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc6280018d0 tx=0x7fc618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e09a3dbfdcc8b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:34.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.637+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628037070 con 0x7fc618003b90 2024-04-15T15:20:34.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.637+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6280052d0 con 0x7fc618003b90 2024-04-15T15:20:34.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.637+0000 7fc634a00640 1 -- 172.21.15.49:0/3020495260 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6280014f0 con 0x7fc618003b90 2024-04-15T15:20:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.638+0000 7fc637200640 1 -- 172.21.15.49:0/3020495260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 msgr2=0x7fc618003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.638+0000 7fc637200640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc618003f90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc6280018d0 tx=0x7fc618005830 comp rx=0 tx=0).stop 2024-04-15T15:20:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.638+0000 7fc637200640 1 -- 172.21.15.49:0/3020495260 shutdown_connections 2024-04-15T15:20:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.638+0000 7fc637200640 1 --2- 172.21.15.49:0/3020495260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc618003f90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.638+0000 7fc637200640 1 -- 172.21.15.49:0/3020495260 >> 172.21.15.49:0/3020495260 conn(0x7fc6380b23a0 msgr2=0x7fc6380b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.638+0000 7fc637200640 1 -- 172.21.15.49:0/3020495260 shutdown_connections 2024-04-15T15:20:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.638+0000 7fc637200640 1 -- 172.21.15.49:0/3020495260 wait complete. 2024-04-15T15:20:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.640+0000 7fc637200640 1 Processor -- start 2024-04-15T15:20:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.640+0000 7fc637200640 1 -- start start 2024-04-15T15:20:34.644 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-04-15T15:20:34.640+0000 7f4225800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:20:34.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.641+0000 7fc637200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc63814c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.641+0000 7fc637200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc63814d340 0x7fc63814f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.641+0000 7fc637200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc63814cd80 0x7fc638157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.641+0000 7fc637200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6380b5e40 con 0x7fc618003b90 2024-04-15T15:20:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.641+0000 7fc637200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6380b1090 con 0x7fc63814cd80 2024-04-15T15:20:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.641+0000 7fc637200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc6380b5fc0 con 0x7fc63814d340 2024-04-15T15:20:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.641+0000 7fc635e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc63814c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.641+0000 7fc635e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc63814c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41380/0 (socket says 172.21.15.49:41380) 2024-04-15T15:20:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.641+0000 7fc635e00640 1 -- 172.21.15.49:0/163269062 learned_addr learned my addr 172.21.15.49:0/163269062 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.642+0000 7fc636800640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc63814cd80 0x7fc638157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.642+0000 7fc635e00640 1 -- 172.21.15.49:0/163269062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc63814d340 msgr2=0x7fc63814f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.642+0000 7fc635400640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc63814d340 0x7fc63814f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.642+0000 7fc635e00640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc63814d340 0x7fc63814f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.642+0000 7fc635e00640 1 -- 172.21.15.49:0/163269062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc63814cd80 msgr2=0x7fc638157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.642+0000 7fc635e00640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc63814cd80 0x7fc638157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.642+0000 7fc635e00640 1 -- 172.21.15.49:0/163269062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63814e390 con 0x7fc618003b90 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.642+0000 7fc636800640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc63814cd80 0x7fc638157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.642+0000 7fc635e00640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc63814c5e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc6280018d0 tx=0x7fc6280012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.643+0000 7fc62e000640 1 -- 172.21.15.49:0/163269062 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628042020 con 0x7fc618003b90 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.643+0000 7fc62e000640 1 -- 172.21.15.49:0/163269062 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc628038040 con 0x7fc618003b90 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.643+0000 7fc62e000640 1 -- 172.21.15.49:0/163269062 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628037070 con 0x7fc618003b90 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.643+0000 7fc637200640 1 -- 172.21.15.49:0/163269062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63814e670 con 0x7fc618003b90 2024-04-15T15:20:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.643+0000 7fc637200640 1 -- 172.21.15.49:0/163269062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63814ec30 con 0x7fc618003b90 2024-04-15T15:20:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.646+0000 7fc62e000640 1 -- 172.21.15.49:0/163269062 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc628045020 con 0x7fc618003b90 2024-04-15T15:20:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.647+0000 7fc62e000640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc61406c9b0 0x7fc61406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.647+0000 7fc62e000640 1 -- 172.21.15.49:0/163269062 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fc6280ad340 con 0x7fc618003b90 2024-04-15T15:20:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.647+0000 7fc635400640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc61406c9b0 0x7fc61406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.647+0000 7fc63fd9c740 1 Processor -- start 2024-04-15T15:20:34.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.647+0000 7fc63fd9c740 1 -- start start 2024-04-15T15:20:34.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.648+0000 7fc635400640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc61406c9b0 0x7fc61406ee70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc620000f80 tx=0x7fc620009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:34.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.648+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204c77e0 0x5561204e6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.648+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.648+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204a0a00 0x55612049e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.648+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5561204059c0 con 0x5561204a0a00 2024-04-15T15:20:34.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.648+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5561203ac560 con 0x5561204c7bc0 2024-04-15T15:20:34.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.648+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x556120401980 con 0x5561204c77e0 2024-04-15T15:20:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.648+0000 7fc636800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204a0a00 0x55612049e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc636800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204a0a00 0x55612049e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41396/0 (socket says 172.21.15.49:41396) 2024-04-15T15:20:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc636800640 1 -- 172.21.15.49:0/3687800989 learned_addr learned my addr 172.21.15.49:0/3687800989 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc635e00640 1 --2- 172.21.15.49:0/3687800989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204c77e0 0x5561204e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc635400640 1 --2- 172.21.15.49:0/3687800989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc636800640 1 -- 172.21.15.49:0/3687800989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204c77e0 msgr2=0x5561204e6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc636800640 1 --2- 172.21.15.49:0/3687800989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204c77e0 0x5561204e6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc636800640 1 -- 172.21.15.49:0/3687800989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 msgr2=0x5561204a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc636800640 1 --2- 172.21.15.49:0/3687800989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc636800640 1 -- 172.21.15.49:0/3687800989 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561201182d0 con 0x5561204a0a00 2024-04-15T15:20:34.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc635e00640 1 --2- 172.21.15.49:0/3687800989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204c77e0 0x5561204e6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc636800640 1 --2- 172.21.15.49:0/3687800989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204a0a00 0x55612049e390 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc638157140 tx=0x7fc630009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b07a3be793d98c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.649+0000 7fc626a00640 1 -- 172.21.15.49:0/3687800989 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630011930 con 0x5561204a0a00 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc626a00640 1 -- 172.21.15.49:0/3687800989 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63001c070 con 0x5561204a0a00 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc626a00640 1 -- 172.21.15.49:0/3687800989 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630011d30 con 0x5561204a0a00 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc63fd9c740 1 -- 172.21.15.49:0/3687800989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204a0a00 msgr2=0x55612049e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/3687800989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204a0a00 0x55612049e390 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc638157140 tx=0x7fc630009190 comp rx=0 tx=0).stop 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc63fd9c740 1 -- 172.21.15.49:0/3687800989 shutdown_connections 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/3687800989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204a0a00 0x55612049e390 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/3687800989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/3687800989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204c77e0 0x5561204e6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc63fd9c740 1 -- 172.21.15.49:0/3687800989 >> 172.21.15.49:0/3687800989 conn(0x55612049caa0 msgr2=0x5561204e5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:34.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc63fd9c740 1 -- 172.21.15.49:0/3687800989 shutdown_connections 2024-04-15T15:20:34.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.650+0000 7fc63fd9c740 1 -- 172.21.15.49:0/3687800989 wait complete. 2024-04-15T15:20:34.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.651+0000 7fc63fd9c740 1 -- start start 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.651+0000 7fc63fd9c740 1 Processor -- start 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.652+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.652+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.652+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204e0f30 0x5561204e0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.652+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55612049a8d0 con 0x5561204c77e0 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.652+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5561203bf920 con 0x5561204c7bc0 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.652+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x5561204a0e80 con 0x5561204e0f30 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.652+0000 7fc63fd9c740 10 client.0 _ll_register_callbacks cb 0x5561203bf310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.652+0000 7fc635e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc635e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204c1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41410/0 (socket says 172.21.15.49:41410) 2024-04-15T15:20:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc635e00640 1 -- 172.21.15.49:0/896023208 learned_addr learned my addr 172.21.15.49:0/896023208 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc635e00640 1 -- 172.21.15.49:0/896023208 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204e0f30 msgr2=0x5561204e0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc635e00640 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204e0f30 0x5561204e0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc635e00640 1 -- 172.21.15.49:0/896023208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 msgr2=0x5561204f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc635e00640 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc635e00640 1 -- 172.21.15.49:0/896023208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561201e26b0 con 0x5561204c77e0 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc635e00640 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204c1ab0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc6280835c0 tx=0x7fc62807afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd745d81efb7d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc61f400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc61f400640 1 -- 172.21.15.49:0/896023208 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6280052d0 con 0x5561204c77e0 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc61f400640 1 -- 172.21.15.49:0/896023208 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fc628083660 con 0x5561204c77e0 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.653+0000 7fc63fd9c740 1 -- 172.21.15.49:0/896023208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc63001c070 con 0x5561204c77e0 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc61f400640 1 -- 172.21.15.49:0/896023208 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc628005470 con 0x5561204c77e0 2024-04-15T15:20:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc63fd9c740 10 client.5484 fetch_fsmap learned FSMap version 23 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc63fd9c740 1 -- 172.21.15.49:0/896023208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5561204dfce0 con 0x5561204c77e0 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc61f400640 1 -- 172.21.15.49:0/896023208 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63001c070 con 0x5561204c77e0 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc61f400640 1 -- 172.21.15.49:0/896023208 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fc6280ab170 con 0x5561204c77e0 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc63fd9c740 10 client.5484 fetch_fsmap finished waiting for FSMap version 23 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc63fd9c740 10 client.5484 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc63fd9c740 20 client.5484 populate_metadata read hostname 'smithi049' 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.49:6836/3168788676,v1:172.21.15.49:6837/3168788676] conn(0x5561204ce3b0 0x5561204f5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc63fd9c740 4 client.5484 mds_command: new command op to 5445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc63fd9c740 1 -- 172.21.15.49:0/896023208 --> [v2:172.21.15.49:6836/3168788676,v1:172.21.15.49:6837/3168788676] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561203abb70 con 0x5561204ce3b0 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.654+0000 7fc635400640 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.49:6836/3168788676,v1:172.21.15.49:6837/3168788676] conn(0x5561204ce3b0 0x5561204f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.655+0000 7fc635400640 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.49:6836/3168788676,v1:172.21.15.49:6837/3168788676] conn(0x5561204ce3b0 0x5561204f5740 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382de4dd39623d74 server_cookie=5fce592b494871f2 in_seq=0 out_seq=0 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.655+0000 7fc61f400640 10 client.5484 ms_handle_connect on v2:172.21.15.49:6836/3168788676 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc61f400640 1 -- 172.21.15.49:0/896023208 <== mds.0 v2:172.21.15.49:6836/3168788676 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561203abb70 con 0x5561204ce3b0 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc61f400640 10 client.5484 handle_command_reply: tid=0 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 client.5484 shutdown 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 -- 172.21.15.49:0/896023208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 msgr2=0x5561204c1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204c1ab0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc6280835c0 tx=0x7fc62807afd0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 -- 172.21.15.49:0/896023208 shutdown_connections 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.49:6836/3168788676,v1:172.21.15.49:6837/3168788676] conn(0x5561204ce3b0 0x5561204f5740 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204e0f30 0x5561204e0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/896023208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204c1ab0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 -- 172.21.15.49:0/896023208 >> 172.21.15.49:0/896023208 conn(0x5561204a0a00 msgr2=0x5561204996e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:34.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc61f400640 0 client.5484 ms_handle_reset on v2:172.21.15.49:6836/3168788676 2024-04-15T15:20:34.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 -- 172.21.15.49:0/896023208 shutdown_connections 2024-04-15T15:20:34.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.656+0000 7fc63fd9c740 1 -- 172.21.15.49:0/896023208 wait complete. 2024-04-15T15:20:34.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.657+0000 7fc63fd9c740 20 client.5484 trim_cache size 0 max 16384 2024-04-15T15:20:34.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.676+0000 7fc63fd9c740 1 Processor -- start 2024-04-15T15:20:34.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.676+0000 7fc63fd9c740 1 -- start start 2024-04-15T15:20:34.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204df280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204e0f30 0x5561204e4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6381fc2e0 con 0x5561204c77e0 2024-04-15T15:20:34.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc638151b10 con 0x5561204c7bc0 2024-04-15T15:20:34.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc6381fbcc0 con 0x5561204e0f30 2024-04-15T15:20:34.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc635e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204df280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc635e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204df280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41416/0 (socket says 172.21.15.49:41416) 2024-04-15T15:20:34.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc635e00640 1 -- 172.21.15.49:0/566627799 learned_addr learned my addr 172.21.15.49:0/566627799 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:34.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc635e00640 1 -- 172.21.15.49:0/566627799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204e0f30 msgr2=0x5561204e4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc635e00640 1 --2- 172.21.15.49:0/566627799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204e0f30 0x5561204e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc635e00640 1 -- 172.21.15.49:0/566627799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 msgr2=0x5561204e39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc635e00640 1 --2- 172.21.15.49:0/566627799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.677+0000 7fc635e00640 1 -- 172.21.15.49:0/566627799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612049acf0 con 0x5561204c77e0 2024-04-15T15:20:34.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.678+0000 7fc635e00640 1 --2- 172.21.15.49:0/566627799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204df280 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc62807ac10 tx=0x7fc628040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab9fa3784cf76db server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.678+0000 7fc626000640 1 -- 172.21.15.49:0/566627799 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6280ab170 con 0x5561204c77e0 2024-04-15T15:20:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.678+0000 7fc626000640 1 -- 172.21.15.49:0/566627799 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc628077580 con 0x5561204c77e0 2024-04-15T15:20:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.678+0000 7fc626000640 1 -- 172.21.15.49:0/566627799 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628083860 con 0x5561204c77e0 2024-04-15T15:20:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.679+0000 7fc63fd9c740 1 -- 172.21.15.49:0/566627799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 msgr2=0x5561204df280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.679+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/566627799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204df280 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc62807ac10 tx=0x7fc628040000 comp rx=0 tx=0).stop 2024-04-15T15:20:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.679+0000 7fc63fd9c740 1 -- 172.21.15.49:0/566627799 shutdown_connections 2024-04-15T15:20:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.679+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/566627799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204e0f30 0x5561204e4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.679+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/566627799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204e39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.679+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/566627799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204c77e0 0x5561204df280 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.679+0000 7fc63fd9c740 1 -- 172.21.15.49:0/566627799 >> 172.21.15.49:0/566627799 conn(0x5561204a0a00 msgr2=0x5561204f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:34.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.679+0000 7fc63fd9c740 1 -- 172.21.15.49:0/566627799 shutdown_connections 2024-04-15T15:20:34.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.679+0000 7fc63fd9c740 1 -- 172.21.15.49:0/566627799 wait complete. 2024-04-15T15:20:34.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.679+0000 7fc63fd9c740 1 -- start start 2024-04-15T15:20:34.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.680+0000 7fc63fd9c740 1 Processor -- start 2024-04-15T15:20:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204c77e0 0x55612055f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204e4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc63fd9c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204e0f30 0x5561204e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5561204e1850 con 0x5561204e0f30 2024-04-15T15:20:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55612049a8d0 con 0x5561204c7bc0 2024-04-15T15:20:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc63fd9c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x5561204e8730 con 0x5561204c77e0 2024-04-15T15:20:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc63fd9c740 10 client.0 _ll_register_callbacks cb 0x7fc62807b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:20:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc636800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204e0f30 0x5561204e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc636800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204e0f30 0x5561204e4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41432/0 (socket says 172.21.15.49:41432) 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc636800640 1 -- 172.21.15.49:0/250926777 learned_addr learned my addr 172.21.15.49:0/250926777 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc636800640 1 -- 172.21.15.49:0/250926777 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204c77e0 msgr2=0x55612055f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc636800640 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204c77e0 0x55612055f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc636800640 1 -- 172.21.15.49:0/250926777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 msgr2=0x5561204e4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc636800640 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.681+0000 7fc636800640 1 -- 172.21.15.49:0/250926777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561204e5710 con 0x5561204e0f30 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.682+0000 7fc636800640 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204e0f30 0x5561204e4b90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc63815db50 tx=0x7fc630009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b22e1af7193d1c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.682+0000 7fc61f400640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.682+0000 7fc61f400640 1 -- 172.21.15.49:0/250926777 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630029060 con 0x5561204e0f30 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.682+0000 7fc61f400640 1 -- 172.21.15.49:0/250926777 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fc630004230 con 0x5561204e0f30 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.682+0000 7fc63fd9c740 1 -- 172.21.15.49:0/250926777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc628077580 con 0x5561204e0f30 2024-04-15T15:20:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.682+0000 7fc61f400640 1 -- 172.21.15.49:0/250926777 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc63002f050 con 0x5561204e0f30 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.682+0000 7fc63fd9c740 10 client.5490 fetch_fsmap learned FSMap version 23 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.682+0000 7fc63fd9c740 1 -- 172.21.15.49:0/250926777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5561204e5cd0 con 0x5561204e0f30 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.683+0000 7fc61f400640 1 -- 172.21.15.49:0/250926777 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc628077580 con 0x5561204e0f30 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.683+0000 7fc61f400640 1 -- 172.21.15.49:0/250926777 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fc630032020 con 0x5561204e0f30 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.683+0000 7fc63fd9c740 10 client.5490 fetch_fsmap finished waiting for FSMap version 23 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.683+0000 7fc63fd9c740 10 client.5490 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.683+0000 7fc63fd9c740 20 client.5490 populate_metadata read hostname 'smithi049' 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.683+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.49:6836/3168788676,v1:172.21.15.49:6837/3168788676] conn(0x5561204cd860 0x5561204e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.683+0000 7fc63fd9c740 4 client.5490 mds_command: new command op to 5445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.683+0000 7fc63fd9c740 1 -- 172.21.15.49:0/250926777 --> [v2:172.21.15.49:6836/3168788676,v1:172.21.15.49:6837/3168788676] -- command(tid 0: {"prefix": "status"}) -- 0x5561204e9100 con 0x5561204cd860 2024-04-15T15:20:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.683+0000 7fc635e00640 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.49:6836/3168788676,v1:172.21.15.49:6837/3168788676] conn(0x5561204cd860 0x5561204e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc635e00640 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.49:6836/3168788676,v1:172.21.15.49:6837/3168788676] conn(0x5561204cd860 0x5561204e19d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d4e0e896a01f68d server_cookie=b1bee6d2114ef237 in_seq=0 out_seq=0 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc61f400640 10 client.5490 ms_handle_connect on v2:172.21.15.49:6836/3168788676 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc61f400640 1 -- 172.21.15.49:0/250926777 <== mds.0 v2:172.21.15.49:6836/3168788676 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5561204e9100 con 0x5561204cd860 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc61f400640 10 client.5490 handle_command_reply: tid=0 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 client.5490 shutdown 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 -- 172.21.15.49:0/250926777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204e0f30 msgr2=0x5561204e4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204e0f30 0x5561204e4b90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc63815db50 tx=0x7fc630009500 comp rx=0 tx=0).stop 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 -- 172.21.15.49:0/250926777 shutdown_connections 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561204e0f30 0x5561204e4b90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561204c7bc0 0x5561204e4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.49:6836/3168788676,v1:172.21.15.49:6837/3168788676] conn(0x5561204cd860 0x5561204e19d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 --2- 172.21.15.49:0/250926777 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5561204c77e0 0x55612055f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 -- 172.21.15.49:0/250926777 >> 172.21.15.49:0/250926777 conn(0x5561204a0a00 msgr2=0x5561204f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc61f400640 0 client.5490 ms_handle_reset on v2:172.21.15.49:6836/3168788676 2024-04-15T15:20:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 -- 172.21.15.49:0/250926777 shutdown_connections 2024-04-15T15:20:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 1 -- 172.21.15.49:0/250926777 wait complete. 2024-04-15T15:20:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.684+0000 7fc63fd9c740 20 client.5490 trim_cache size 0 max 16384 2024-04-15T15:20:34.691 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:20:34.691 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c", 2024-04-15T15:20:34.691 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-04-15T15:20:34.691 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 5445, 2024-04-15T15:20:34.692 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-04-15T15:20:34.692 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-04-15T15:20:34.692 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-04-15T15:20:34.692 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 5.0464085650000001, 2024-04-15T15:20:34.692 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2024-04-15T15:20:34.692 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 36, 2024-04-15T15:20:34.692 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 36, 2024-04-15T15:20:34.692 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 5.9695121850000001 2024-04-15T15:20:34.692 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:20:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 -- 172.21.15.49:0/163269062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc61406c9b0 msgr2=0x7fc61406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc61406c9b0 0x7fc61406ee70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc620000f80 tx=0x7fc620009040 comp rx=0 tx=0).stop 2024-04-15T15:20:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 -- 172.21.15.49:0/163269062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 msgr2=0x7fc63814c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc63814c5e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc6280018d0 tx=0x7fc6280012d0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 -- 172.21.15.49:0/163269062 shutdown_connections 2024-04-15T15:20:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc61406c9b0 0x7fc61406ee70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc63814d340 0x7fc63814f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc63814cd80 0x7fc638157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 --2- 172.21.15.49:0/163269062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc618003b90 0x7fc63814c5e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:34.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 -- 172.21.15.49:0/163269062 >> 172.21.15.49:0/163269062 conn(0x7fc6380b23a0 msgr2=0x7fc63815abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:34.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 -- 172.21.15.49:0/163269062 shutdown_connections 2024-04-15T15:20:34.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:34.685+0000 7fc637200640 1 -- 172.21.15.49:0/163269062 wait complete. 2024-04-15T15:20:34.703 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-15T15:20:34.703 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:20:34.830 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:20:34.964 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-15T15:20:34.964 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:20:34.965 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:20:34.965 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:35.026 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:20:35.026 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:35.083 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:35.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.316+0000 7f6d43200640 1 Processor -- start 2024-04-15T15:20:35.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.317+0000 7f6d43200640 1 -- start start 2024-04-15T15:20:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.318+0000 7f6d43200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d44162020 0x7f6d44162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.318+0000 7f6d43200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6d440cf110 0x7f6d440d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.318+0000 7f6d43200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6d440d1ac0 0x7f6d440b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.318+0000 7f6d43200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f6d440d1f50 con 0x7f6d440d1ac0 2024-04-15T15:20:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.318+0000 7f6d43200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f6d44163440 con 0x7f6d440cf110 2024-04-15T15:20:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.318+0000 7f6d43200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6d440b7b30 con 0x7f6d44162020 2024-04-15T15:20:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.318+0000 7f6d42800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6d440d1ac0 0x7f6d440b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.318+0000 7f6d42800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6d440d1ac0 0x7f6d440b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41448/0 (socket says 172.21.15.49:41448) 2024-04-15T15:20:35.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.318+0000 7f6d42800640 1 -- 172.21.15.49:0/1101528126 learned_addr learned my addr 172.21.15.49:0/1101528126 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:35.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.319+0000 7f6d41e00640 1 --2- 172.21.15.49:0/1101528126 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d44162020 0x7f6d44162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.319+0000 7f6d39400640 1 --2- 172.21.15.49:0/1101528126 >> v2:172.21.15.107:3300/0 conn(0x7f6d440cf110 0x7f6d440d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.319+0000 7f6d42800640 1 -- 172.21.15.49:0/1101528126 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d44162020 msgr2=0x7f6d44162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:35.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.319+0000 7f6d42800640 1 --2- 172.21.15.49:0/1101528126 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d44162020 0x7f6d44162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.319+0000 7f6d42800640 1 -- 172.21.15.49:0/1101528126 >> v2:172.21.15.107:3300/0 conn(0x7f6d440cf110 msgr2=0x7f6d440d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:35.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.319+0000 7f6d42800640 1 --2- 172.21.15.49:0/1101528126 >> v2:172.21.15.107:3300/0 conn(0x7f6d440cf110 0x7f6d440d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.319+0000 7f6d42800640 1 -- 172.21.15.49:0/1101528126 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d440d0090 con 0x7f6d440d1ac0 2024-04-15T15:20:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.320+0000 7f6d42800640 1 --2- 172.21.15.49:0/1101528126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440d1ac0 0x7f6d440b62a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6d440b7410 tx=0x7f6d3c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85888c4ff947d3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.320+0000 7f6d41400640 1 -- 172.21.15.49:0/1101528126 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c02ca70 con 0x7f6d440d1ac0 2024-04-15T15:20:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.321+0000 7f6d41400640 1 -- 172.21.15.49:0/1101528126 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d3c02cc10 con 0x7f6d440d1ac0 2024-04-15T15:20:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.321+0000 7f6d41400640 1 -- 172.21.15.49:0/1101528126 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0343f0 con 0x7f6d440d1ac0 2024-04-15T15:20:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.321+0000 7f6d43200640 1 -- 172.21.15.49:0/1101528126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440d1ac0 msgr2=0x7f6d440b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.321+0000 7f6d43200640 1 --2- 172.21.15.49:0/1101528126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440d1ac0 0x7f6d440b62a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6d440b7410 tx=0x7f6d3c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.321+0000 7f6d43200640 1 -- 172.21.15.49:0/1101528126 shutdown_connections 2024-04-15T15:20:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.321+0000 7f6d43200640 1 --2- 172.21.15.49:0/1101528126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440d1ac0 0x7f6d440b62a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.321+0000 7f6d43200640 1 --2- 172.21.15.49:0/1101528126 >> v2:172.21.15.107:3300/0 conn(0x7f6d440cf110 0x7f6d440d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.321+0000 7f6d43200640 1 --2- 172.21.15.49:0/1101528126 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d44162020 0x7f6d44162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.321+0000 7f6d43200640 1 -- 172.21.15.49:0/1101528126 >> 172.21.15.49:0/1101528126 conn(0x7f6d4406cf00 msgr2=0x7f6d4406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.321+0000 7f6d43200640 1 -- 172.21.15.49:0/1101528126 shutdown_connections 2024-04-15T15:20:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.322+0000 7f6d43200640 1 -- 172.21.15.49:0/1101528126 wait complete. 2024-04-15T15:20:35.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.323+0000 7f6d43200640 1 Processor -- start 2024-04-15T15:20:35.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.323+0000 7f6d43200640 1 -- start start 2024-04-15T15:20:35.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.324+0000 7f6d43200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440cf110 0x7f6d440c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.324+0000 7f6d43200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d440d1ac0 0x7f6d440c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.324+0000 7f6d43200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d44162020 0x7f6d4415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.324+0000 7f6d43200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6d440cfdb0 con 0x7f6d440cf110 2024-04-15T15:20:35.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.324+0000 7f6d43200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6d440cf880 con 0x7f6d44162020 2024-04-15T15:20:35.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.324+0000 7f6d43200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6d440d2b90 con 0x7f6d440d1ac0 2024-04-15T15:20:35.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.325+0000 7f6d41e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440cf110 0x7f6d440c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.325+0000 7f6d41e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440cf110 0x7f6d440c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41464/0 (socket says 172.21.15.49:41464) 2024-04-15T15:20:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.325+0000 7f6d41e00640 1 -- 172.21.15.49:0/1568204166 learned_addr learned my addr 172.21.15.49:0/1568204166 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.325+0000 7f6d42800640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d44162020 0x7f6d4415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.325+0000 7f6d39400640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d440d1ac0 0x7f6d440c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.325+0000 7f6d41e00640 1 -- 172.21.15.49:0/1568204166 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d440d1ac0 msgr2=0x7f6d440c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.325+0000 7f6d41e00640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d440d1ac0 0x7f6d440c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.325+0000 7f6d41e00640 1 -- 172.21.15.49:0/1568204166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d44162020 msgr2=0x7f6d4415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.325+0000 7f6d41e00640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d44162020 0x7f6d4415e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.325+0000 7f6d41e00640 1 -- 172.21.15.49:0/1568204166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d440c8d30 con 0x7f6d440cf110 2024-04-15T15:20:35.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.326+0000 7f6d41e00640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440cf110 0x7f6d440c87f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400c7f0 tx=0x7f6d3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:35.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.326+0000 7f6d3aa00640 1 -- 172.21.15.49:0/1568204166 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34015070 con 0x7f6d440cf110 2024-04-15T15:20:35.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.326+0000 7f6d3aa00640 1 -- 172.21.15.49:0/1568204166 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d34004240 con 0x7f6d440cf110 2024-04-15T15:20:35.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.326+0000 7f6d3aa00640 1 -- 172.21.15.49:0/1568204166 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34007620 con 0x7f6d440cf110 2024-04-15T15:20:35.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.326+0000 7f6d43200640 1 -- 172.21.15.49:0/1568204166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d441603e0 con 0x7f6d440cf110 2024-04-15T15:20:35.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.326+0000 7f6d43200640 1 -- 172.21.15.49:0/1568204166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d44160c80 con 0x7f6d440cf110 2024-04-15T15:20:35.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.327+0000 7f6d43200640 1 -- 172.21.15.49:0/1568204166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d440755c0 con 0x7f6d440cf110 2024-04-15T15:20:35.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.328+0000 7f6d3aa00640 1 -- 172.21.15.49:0/1568204166 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6d340079d0 con 0x7f6d440cf110 2024-04-15T15:20:35.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.329+0000 7f6d3aa00640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6d2006c980 0x7f6d2006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.329+0000 7f6d3aa00640 1 -- 172.21.15.49:0/1568204166 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f6d3408bef0 con 0x7f6d440cf110 2024-04-15T15:20:35.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.330+0000 7f6d39400640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6d2006c980 0x7f6d2006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.330+0000 7f6d39400640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6d2006c980 0x7f6d2006ee40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d440ce3a0 tx=0x7f6d2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:35.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.334+0000 7f6d3aa00640 1 -- 172.21.15.49:0/1568204166 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d3405a0c0 con 0x7f6d440cf110 2024-04-15T15:20:35.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.540+0000 7f6d43200640 1 -- 172.21.15.49:0/1568204166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d440d2e60 con 0x7f6d440cf110 2024-04-15T15:20:35.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.542+0000 7f6d3aa00640 1 -- 172.21.15.49:0/1568204166 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16798 (secure 0 0 0) 0x7f6d340599c0 con 0x7f6d440cf110 2024-04-15T15:20:35.544 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:35.544 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:28.618835+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6837/1023024717":"2024-04-16T15:20:28.618766+0000","172.21.15.49:6836/1023024717":"2024-04-16T15:20:28.618766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 -- 172.21.15.49:0/1568204166 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6d2006c980 msgr2=0x7f6d2006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6d2006c980 0x7f6d2006ee40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d440ce3a0 tx=0x7f6d2c008040 comp rx=0 tx=0).stop 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 -- 172.21.15.49:0/1568204166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440cf110 msgr2=0x7f6d440c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440cf110 0x7f6d440c87f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400c7f0 tx=0x7f6d3400ccb0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 -- 172.21.15.49:0/1568204166 shutdown_connections 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6d2006c980 0x7f6d2006ee40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d440d1ac0 0x7f6d440c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d44162020 0x7f6d4415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 --2- 172.21.15.49:0/1568204166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d440cf110 0x7f6d440c87f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 -- 172.21.15.49:0/1568204166 >> 172.21.15.49:0/1568204166 conn(0x7f6d4406cf00 msgr2=0x7f6d4415cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 -- 172.21.15.49:0/1568204166 shutdown_connections 2024-04-15T15:20:35.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.545+0000 7f6d43200640 1 -- 172.21.15.49:0/1568204166 wait complete. 2024-04-15T15:20:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:58.063498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:59.009342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:35.567 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:35.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.804+0000 7effde400640 1 Processor -- start 2024-04-15T15:20:35.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.804+0000 7effde400640 1 -- start start 2024-04-15T15:20:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.805+0000 7effde400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7effd80d26b0 0x7effd80b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.805+0000 7effde400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7effd80b7e80 0x7effd80d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.806+0000 7effde400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effd80cfef0 con 0x7effd8161f80 2024-04-15T15:20:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.806+0000 7effde400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7effd80d2bb0 con 0x7effd80b7e80 2024-04-15T15:20:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.806+0000 7effde400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7effd80d2e90 con 0x7effd80d26b0 2024-04-15T15:20:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.806+0000 7effdd000640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7effd8161f80 0x7effd80cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60090/0 (socket says 172.21.15.49:60090) 2024-04-15T15:20:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.806+0000 7effdd000640 1 -- 172.21.15.49:0/4264216652 learned_addr learned my addr 172.21.15.49:0/4264216652 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.806+0000 7effdda00640 1 --2- 172.21.15.49:0/4264216652 >> v2:172.21.15.49:3300/0 conn(0x7effd80b7e80 0x7effd80d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.806+0000 7effd7e00640 1 --2- 172.21.15.49:0/4264216652 >> v2:172.21.15.107:3300/0 conn(0x7effd80d26b0 0x7effd80b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.806+0000 7effd7400640 1 -- 172.21.15.49:0/4264216652 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2116499752 0 0) 0x7effd80cfef0 con 0x7effd8161f80 2024-04-15T15:20:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.806+0000 7effd7400640 1 -- 172.21.15.49:0/4264216652 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effbc003520 con 0x7effd8161f80 2024-04-15T15:20:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.807+0000 7effdda00640 1 -- 172.21.15.49:0/4264216652 >> v2:172.21.15.107:3300/0 conn(0x7effd80d26b0 msgr2=0x7effd80b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.807+0000 7effdda00640 1 --2- 172.21.15.49:0/4264216652 >> v2:172.21.15.107:3300/0 conn(0x7effd80d26b0 0x7effd80b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.807+0000 7effdda00640 1 -- 172.21.15.49:0/4264216652 >> v1:172.21.15.49:6789/0 conn(0x7effd8161f80 legacy=0x7effd80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.807+0000 7effdda00640 1 -- 172.21.15.49:0/4264216652 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd80d3170 con 0x7effd80b7e80 2024-04-15T15:20:35.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.808+0000 7effdda00640 1 --2- 172.21.15.49:0/4264216652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80b7e80 0x7effd80d1580 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7effd80d2400 tx=0x7effd002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513fdecf2b7baec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:35.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.808+0000 7effd7400640 1 -- 172.21.15.49:0/4264216652 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0039070 con 0x7effd80b7e80 2024-04-15T15:20:35.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.808+0000 7effd7400640 1 -- 172.21.15.49:0/4264216652 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd002cc80 con 0x7effd80b7e80 2024-04-15T15:20:35.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.808+0000 7effd7400640 1 -- 172.21.15.49:0/4264216652 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0034640 con 0x7effd80b7e80 2024-04-15T15:20:35.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.809+0000 7effde400640 1 -- 172.21.15.49:0/4264216652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80b7e80 msgr2=0x7effd80d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:35.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.809+0000 7effde400640 1 --2- 172.21.15.49:0/4264216652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80b7e80 0x7effd80d1580 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7effd80d2400 tx=0x7effd002bed0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.809+0000 7effde400640 1 -- 172.21.15.49:0/4264216652 shutdown_connections 2024-04-15T15:20:35.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.809+0000 7effde400640 1 --2- 172.21.15.49:0/4264216652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80b7e80 0x7effd80d1580 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.809+0000 7effde400640 1 --2- 172.21.15.49:0/4264216652 >> v2:172.21.15.107:3300/0 conn(0x7effd80d26b0 0x7effd80b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.809+0000 7effde400640 1 -- 172.21.15.49:0/4264216652 >> 172.21.15.49:0/4264216652 conn(0x7effd806ce40 msgr2=0x7effd806d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:35.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.809+0000 7effde400640 1 -- 172.21.15.49:0/4264216652 shutdown_connections 2024-04-15T15:20:35.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.809+0000 7effde400640 1 -- 172.21.15.49:0/4264216652 wait complete. 2024-04-15T15:20:35.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.810+0000 7effde400640 1 Processor -- start 2024-04-15T15:20:35.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.811+0000 7effde400640 1 -- start start 2024-04-15T15:20:35.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effde400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd80b7e80 0x7effd81502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effde400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80d26b0 0x7effd8158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effde400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd8161f80 0x7effd8159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effde400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7effd80d1ac0 con 0x7effd80d26b0 2024-04-15T15:20:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effde400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7effd8162360 con 0x7effd8161f80 2024-04-15T15:20:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effde400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7effd80d1c40 con 0x7effd80b7e80 2024-04-15T15:20:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effd7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80d26b0 0x7effd8158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effd7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80d26b0 0x7effd8158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41482/0 (socket says 172.21.15.49:41482) 2024-04-15T15:20:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effd7e00640 1 -- 172.21.15.49:0/1418590077 learned_addr learned my addr 172.21.15.49:0/1418590077 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effdd000640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd80b7e80 0x7effd81502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effd7e00640 1 -- 172.21.15.49:0/1418590077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd80b7e80 msgr2=0x7effd81502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effd7e00640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd80b7e80 0x7effd81502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effd7e00640 1 -- 172.21.15.49:0/1418590077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd8161f80 msgr2=0x7effd8159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effd7e00640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd8161f80 0x7effd8159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.812+0000 7effd7e00640 1 -- 172.21.15.49:0/1418590077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd8158bd0 con 0x7effd80d26b0 2024-04-15T15:20:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.813+0000 7effd7e00640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80d26b0 0x7effd8158180 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7effc000d7d0 tx=0x7effc000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.813+0000 7effd4c00640 1 -- 172.21.15.49:0/1418590077 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc000b6c0 con 0x7effd80d26b0 2024-04-15T15:20:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.813+0000 7effd4c00640 1 -- 172.21.15.49:0/1418590077 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc0019070 con 0x7effd80d26b0 2024-04-15T15:20:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.813+0000 7effd4c00640 1 -- 172.21.15.49:0/1418590077 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc000ba80 con 0x7effd80d26b0 2024-04-15T15:20:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.813+0000 7effde400640 1 -- 172.21.15.49:0/1418590077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd8158eb0 con 0x7effd80d26b0 2024-04-15T15:20:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.813+0000 7effde400640 1 -- 172.21.15.49:0/1418590077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd815a930 con 0x7effd80d26b0 2024-04-15T15:20:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.815+0000 7effde400640 1 -- 172.21.15.49:0/1418590077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd8155240 con 0x7effd80d26b0 2024-04-15T15:20:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.815+0000 7effd4c00640 1 -- 172.21.15.49:0/1418590077 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7effc000bc20 con 0x7effd80d26b0 2024-04-15T15:20:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.816+0000 7effd4c00640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7effb406c980 0x7effb406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.816+0000 7effd4c00640 1 -- 172.21.15.49:0/1418590077 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7effc008c0b0 con 0x7effd80d26b0 2024-04-15T15:20:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.817+0000 7effdd000640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7effb406c980 0x7effb406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.817+0000 7effdd000640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7effb406c980 0x7effb406ee40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7effc8000f60 tx=0x7effc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:35.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:35.821+0000 7effd4c00640 1 -- 172.21.15.49:0/1418590077 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effc001c360 con 0x7effd80d26b0 2024-04-15T15:20:36.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.017+0000 7effde400640 1 -- 172.21.15.49:0/1418590077 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7effd8068ef0 con 0x7effd80d26b0 2024-04-15T15:20:36.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.018+0000 7effd4c00640 1 -- 172.21.15.49:0/1418590077 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7effc005a490 con 0x7effd80d26b0 2024-04-15T15:20:36.019 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:20:36.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 -- 172.21.15.49:0/1418590077 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7effb406c980 msgr2=0x7effb406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7effb406c980 0x7effb406ee40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7effc8000f60 tx=0x7effc8009040 comp rx=0 tx=0).stop 2024-04-15T15:20:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 -- 172.21.15.49:0/1418590077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80d26b0 msgr2=0x7effd8158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80d26b0 0x7effd8158180 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7effc000d7d0 tx=0x7effc000dc90 comp rx=0 tx=0).stop 2024-04-15T15:20:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 -- 172.21.15.49:0/1418590077 shutdown_connections 2024-04-15T15:20:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7effb406c980 0x7effb406ee40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd8161f80 0x7effd8159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effd80d26b0 0x7effd8158180 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 --2- 172.21.15.49:0/1418590077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd80b7e80 0x7effd81502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 -- 172.21.15.49:0/1418590077 >> 172.21.15.49:0/1418590077 conn(0x7effd806ce40 msgr2=0x7effd8153750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:36.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 -- 172.21.15.49:0/1418590077 shutdown_connections 2024-04-15T15:20:36.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.020+0000 7effde400640 1 -- 172.21.15.49:0/1418590077 wait complete. 2024-04-15T15:20:36.042 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:36.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.271+0000 7fe26b200640 1 Processor -- start 2024-04-15T15:20:36.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.271+0000 7fe26b200640 1 -- start start 2024-04-15T15:20:36.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.273+0000 7fe26b200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe26c162730 0x7fe26c0b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.273+0000 7fe26b200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26c162dd0 con 0x7fe26c15c680 2024-04-15T15:20:36.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.273+0000 7fe26b200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26c1630b0 con 0x7fe26c0d2b50 2024-04-15T15:20:36.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.273+0000 7fe26b200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fe26c162b90 con 0x7fe26c162730 2024-04-15T15:20:36.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.273+0000 7fe269e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe26c15c680 0x7fe26c15f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60106/0 (socket says 172.21.15.49:60106) 2024-04-15T15:20:36.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.273+0000 7fe269e00640 1 -- 172.21.15.49:0/2237245078 learned_addr learned my addr 172.21.15.49:0/2237245078 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe261400640 1 --2- 172.21.15.49:0/2237245078 >> v2:172.21.15.107:3300/0 conn(0x7fe26c162730 0x7fe26c0b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2763963719 0 0) 0x7fe26c162dd0 con 0x7fe26c15c680 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe250003520 con 0x7fe26c15c680 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2859333186 0 0) 0x7fe250003520 con 0x7fe26c15c680 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 >> v2:172.21.15.107:3300/0 conn(0x7fe26c162730 msgr2=0x7fe26c0b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 --2- 172.21.15.49:0/2237245078 >> v2:172.21.15.107:3300/0 conn(0x7fe26c162730 0x7fe26c0b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 >> v1:172.21.15.107:6789/0 conn(0x7fe26c0d2b50 legacy=0x7fe26c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c163390 con 0x7fe26c15c680 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fe25c0019f0 con 0x7fe26c15c680 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 >> v1:172.21.15.49:6789/0 conn(0x7fe26c15c680 legacy=0x7fe26c15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 --2- 172.21.15.49:0/2237245078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2500039b0 0x7fe250003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.274+0000 7fe269400640 1 --2- 172.21.15.49:0/2237245078 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe250004800 0x7fe250004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe269400640 1 --2- 172.21.15.49:0/2237245078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2500051c0 0x7fe25000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c162b90 con 0x7fe2500039b0 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe25000c060 con 0x7fe2500051c0 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe25000c260 con 0x7fe250004800 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe269e00640 1 --2- 172.21.15.49:0/2237245078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2500039b0 0x7fe250003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe269e00640 1 -- 172.21.15.49:0/2237245078 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe250004800 msgr2=0x7fe250004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe269e00640 1 --2- 172.21.15.49:0/2237245078 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe250004800 0x7fe250004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe269e00640 1 -- 172.21.15.49:0/2237245078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2500051c0 msgr2=0x7fe25000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe269e00640 1 --2- 172.21.15.49:0/2237245078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2500051c0 0x7fe25000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe269e00640 1 -- 172.21.15.49:0/2237245078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe250003520 con 0x7fe2500039b0 2024-04-15T15:20:36.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.275+0000 7fe26a800640 1 -- 172.21.15.49:0/2237245078 reap_dead start 2024-04-15T15:20:36.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.276+0000 7fe269e00640 1 --2- 172.21.15.49:0/2237245078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2500039b0 0x7fe250003dd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00a860 tx=0x7fe250005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad28c5adeb076a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:36.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.277+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c00a480 con 0x7fe2500039b0 2024-04-15T15:20:36.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.277+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25c034630 con 0x7fe2500039b0 2024-04-15T15:20:36.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.277+0000 7fe269400640 1 -- 172.21.15.49:0/2237245078 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c034aa0 con 0x7fe2500039b0 2024-04-15T15:20:36.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.277+0000 7fe26b200640 1 -- 172.21.15.49:0/2237245078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2500039b0 msgr2=0x7fe250003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.277+0000 7fe26b200640 1 --2- 172.21.15.49:0/2237245078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2500039b0 0x7fe250003dd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00a860 tx=0x7fe250005830 comp rx=0 tx=0).stop 2024-04-15T15:20:36.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.277+0000 7fe26b200640 1 -- 172.21.15.49:0/2237245078 shutdown_connections 2024-04-15T15:20:36.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.277+0000 7fe26b200640 1 --2- 172.21.15.49:0/2237245078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2500039b0 0x7fe250003dd0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.277+0000 7fe26b200640 1 -- 172.21.15.49:0/2237245078 >> 172.21.15.49:0/2237245078 conn(0x7fe26c073480 msgr2=0x7fe26c073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:36.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.277+0000 7fe26b200640 1 -- 172.21.15.49:0/2237245078 shutdown_connections 2024-04-15T15:20:36.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.277+0000 7fe26b200640 1 -- 172.21.15.49:0/2237245078 wait complete. 2024-04-15T15:20:36.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.279+0000 7fe26b200640 1 Processor -- start 2024-04-15T15:20:36.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.279+0000 7fe26b200640 1 -- start start 2024-04-15T15:20:36.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.280+0000 7fe26b200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2500039b0 0x7fe26c0b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.280+0000 7fe26b200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe26c0ca340 0x7fe26c0ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.280+0000 7fe26b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe26c1607a0 0x7fe26c160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.280+0000 7fe26b200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c0b6c70 con 0x7fe26c0ca340 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.280+0000 7fe26b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c15ca60 con 0x7fe26c1607a0 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.280+0000 7fe26b200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe26c0b6df0 con 0x7fe2500039b0 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe261400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe26c0ca340 0x7fe26c0ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe261400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe26c0ca340 0x7fe26c0ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41512/0 (socket says 172.21.15.49:41512) 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe261400640 1 -- 172.21.15.49:0/1999574062 learned_addr learned my addr 172.21.15.49:0/1999574062 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe269e00640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2500039b0 0x7fe26c0b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe26a800640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe26c1607a0 0x7fe26c160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe261400640 1 -- 172.21.15.49:0/1999574062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2500039b0 msgr2=0x7fe26c0b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe261400640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2500039b0 0x7fe26c0b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe261400640 1 -- 172.21.15.49:0/1999574062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe26c1607a0 msgr2=0x7fe26c160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe261400640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe26c1607a0 0x7fe26c160c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe261400640 1 -- 172.21.15.49:0/1999574062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c1f8e00 con 0x7fe26c0ca340 2024-04-15T15:20:36.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.281+0000 7fe261400640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe26c0ca340 0x7fe26c0ca7c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe25400e860 tx=0x7fe25400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:36.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.282+0000 7fe262a00640 1 -- 172.21.15.49:0/1999574062 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254017070 con 0x7fe26c0ca340 2024-04-15T15:20:36.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.282+0000 7fe262a00640 1 -- 172.21.15.49:0/1999574062 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25400c8e0 con 0x7fe26c0ca340 2024-04-15T15:20:36.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.282+0000 7fe262a00640 1 -- 172.21.15.49:0/1999574062 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25400ccd0 con 0x7fe26c0ca340 2024-04-15T15:20:36.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.282+0000 7fe26b200640 1 -- 172.21.15.49:0/1999574062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c1f8040 con 0x7fe26c0ca340 2024-04-15T15:20:36.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.282+0000 7fe26b200640 1 -- 172.21.15.49:0/1999574062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c1f8600 con 0x7fe26c0ca340 2024-04-15T15:20:36.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.283+0000 7fe26b200640 1 -- 172.21.15.49:0/1999574062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c0755c0 con 0x7fe26c0ca340 2024-04-15T15:20:36.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.284+0000 7fe262a00640 1 -- 172.21.15.49:0/1999574062 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe25401a070 con 0x7fe26c0ca340 2024-04-15T15:20:36.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.285+0000 7fe262a00640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe24806c980 0x7fe24806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.285+0000 7fe262a00640 1 -- 172.21.15.49:0/1999574062 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fe25408bff0 con 0x7fe26c0ca340 2024-04-15T15:20:36.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.286+0000 7fe269e00640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe24806c980 0x7fe24806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.286+0000 7fe269e00640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe24806c980 0x7fe24806ee40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe25c034000 tx=0x7fe25c036040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:36.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.290+0000 7fe262a00640 1 -- 172.21.15.49:0/1999574062 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe25405a310 con 0x7fe26c0ca340 2024-04-15T15:20:36.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.490+0000 7fe26b200640 1 -- 172.21.15.49:0/1999574062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe26c06dbe0 con 0x7fe26c0ca340 2024-04-15T15:20:36.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.491+0000 7fe262a00640 1 -- 172.21.15.49:0/1999574062 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7fe254059c10 con 0x7fe26c0ca340 2024-04-15T15:20:36.492 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:20:36.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.493+0000 7fe26b200640 1 -- 172.21.15.49:0/1999574062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe24806c980 msgr2=0x7fe24806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.493+0000 7fe26b200640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe24806c980 0x7fe24806ee40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe25c034000 tx=0x7fe25c036040 comp rx=0 tx=0).stop 2024-04-15T15:20:36.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.493+0000 7fe26b200640 1 -- 172.21.15.49:0/1999574062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe26c0ca340 msgr2=0x7fe26c0ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.493+0000 7fe26b200640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe26c0ca340 0x7fe26c0ca7c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe25400e860 tx=0x7fe25400ed20 comp rx=0 tx=0).stop 2024-04-15T15:20:36.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.493+0000 7fe26b200640 1 -- 172.21.15.49:0/1999574062 shutdown_connections 2024-04-15T15:20:36.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.493+0000 7fe26b200640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe24806c980 0x7fe24806ee40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.493+0000 7fe26b200640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe26c1607a0 0x7fe26c160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.493+0000 7fe26b200640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe26c0ca340 0x7fe26c0ca7c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.493+0000 7fe26b200640 1 --2- 172.21.15.49:0/1999574062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2500039b0 0x7fe26c0b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.493+0000 7fe26b200640 1 -- 172.21.15.49:0/1999574062 >> 172.21.15.49:0/1999574062 conn(0x7fe26c073480 msgr2=0x7fe26c0d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:36.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.494+0000 7fe26b200640 1 -- 172.21.15.49:0/1999574062 shutdown_connections 2024-04-15T15:20:36.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.494+0000 7fe26b200640 1 -- 172.21.15.49:0/1999574062 wait complete. 2024-04-15T15:20:36.515 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:36.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.740+0000 7f085de00640 1 Processor -- start 2024-04-15T15:20:36.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.740+0000 7f085de00640 1 -- start start 2024-04-15T15:20:36.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.742+0000 7f085de00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f08581461c0 0x7f0858071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.742+0000 7f085de00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0858072420 0x7f08580ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.742+0000 7f085de00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f08580ce9e0 0x7f0858160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.742+0000 7f085de00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f08580cf1d0 con 0x7f08581461c0 2024-04-15T15:20:36.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.742+0000 7f085de00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f0858072880 con 0x7f08580ce9e0 2024-04-15T15:20:36.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.742+0000 7f085de00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f08580cd080 con 0x7f0858072420 2024-04-15T15:20:36.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.742+0000 7f085ca00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f08581461c0 0x7f0858071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.742+0000 7f085ca00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f08581461c0 0x7f0858071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41516/0 (socket says 172.21.15.49:41516) 2024-04-15T15:20:36.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.742+0000 7f085ca00640 1 -- 172.21.15.49:0/3738628729 learned_addr learned my addr 172.21.15.49:0/3738628729 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:36.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.743+0000 7f0857e00640 1 --2- 172.21.15.49:0/3738628729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0858072420 0x7f08580ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.743+0000 7f085d400640 1 --2- 172.21.15.49:0/3738628729 >> v2:172.21.15.107:3300/0 conn(0x7f08580ce9e0 0x7f0858160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.743+0000 7f085ca00640 1 -- 172.21.15.49:0/3738628729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0858072420 msgr2=0x7f08580ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:36.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.743+0000 7f085ca00640 1 --2- 172.21.15.49:0/3738628729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0858072420 0x7f08580ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.743+0000 7f085ca00640 1 -- 172.21.15.49:0/3738628729 >> v2:172.21.15.107:3300/0 conn(0x7f08580ce9e0 msgr2=0x7f0858160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:36.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.743+0000 7f085ca00640 1 --2- 172.21.15.49:0/3738628729 >> v2:172.21.15.107:3300/0 conn(0x7f08580ce9e0 0x7f0858160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.743+0000 7f085ca00640 1 -- 172.21.15.49:0/3738628729 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08580cd2d0 con 0x7f08581461c0 2024-04-15T15:20:36.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.743+0000 7f085d400640 1 --2- 172.21.15.49:0/3738628729 >> v2:172.21.15.107:3300/0 conn(0x7f08580ce9e0 0x7f0858160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:36.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.744+0000 7f085ca00640 1 --2- 172.21.15.49:0/3738628729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08581461c0 0x7f0858071ee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f08580b5a20 tx=0x7f084c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4df1f1dd591f5d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:36.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.745+0000 7f0857400640 1 -- 172.21.15.49:0/3738628729 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c035020 con 0x7f08581461c0 2024-04-15T15:20:36.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.745+0000 7f0857400640 1 -- 172.21.15.49:0/3738628729 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c031870 con 0x7f08581461c0 2024-04-15T15:20:36.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.745+0000 7f0857400640 1 -- 172.21.15.49:0/3738628729 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c031c40 con 0x7f08581461c0 2024-04-15T15:20:36.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.745+0000 7f085de00640 1 -- 172.21.15.49:0/3738628729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08581461c0 msgr2=0x7f0858071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.745+0000 7f085de00640 1 --2- 172.21.15.49:0/3738628729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08581461c0 0x7f0858071ee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f08580b5a20 tx=0x7f084c02b910 comp rx=0 tx=0).stop 2024-04-15T15:20:36.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.746+0000 7f085de00640 1 -- 172.21.15.49:0/3738628729 shutdown_connections 2024-04-15T15:20:36.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.746+0000 7f085de00640 1 --2- 172.21.15.49:0/3738628729 >> v2:172.21.15.107:3300/0 conn(0x7f08580ce9e0 0x7f0858160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.746+0000 7f085de00640 1 --2- 172.21.15.49:0/3738628729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0858072420 0x7f08580ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.746+0000 7f085de00640 1 --2- 172.21.15.49:0/3738628729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08581461c0 0x7f0858071ee0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.746+0000 7f085de00640 1 -- 172.21.15.49:0/3738628729 >> 172.21.15.49:0/3738628729 conn(0x7f085806b930 msgr2=0x7f085806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:36.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.746+0000 7f085de00640 1 -- 172.21.15.49:0/3738628729 shutdown_connections 2024-04-15T15:20:36.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.746+0000 7f085de00640 1 -- 172.21.15.49:0/3738628729 wait complete. 2024-04-15T15:20:36.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.748+0000 7f085de00640 1 Processor -- start 2024-04-15T15:20:36.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.748+0000 7f085de00640 1 -- start start 2024-04-15T15:20:36.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f085de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0858072420 0x7f08581f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f085de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08580ce9e0 0x7f08580aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f085de00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08581461c0 0x7f08581f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f085de00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f08580cdb70 con 0x7f08580ce9e0 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f085de00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f08580cd890 con 0x7f0858072420 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f085de00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f08580cee70 con 0x7f08581461c0 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f0857e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08580ce9e0 0x7f08580aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f0857e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08580ce9e0 0x7f08580aca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41524/0 (socket says 172.21.15.49:41524) 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f0857e00640 1 -- 172.21.15.49:0/545390860 learned_addr learned my addr 172.21.15.49:0/545390860 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f085d400640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08581461c0 0x7f08581f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f085ca00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0858072420 0x7f08581f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f0857e00640 1 -- 172.21.15.49:0/545390860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08581461c0 msgr2=0x7f08581f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f0857e00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08581461c0 0x7f08581f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f0857e00640 1 -- 172.21.15.49:0/545390860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0858072420 msgr2=0x7f08581f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.749+0000 7f0857e00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0858072420 0x7f08581f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.750+0000 7f0857e00640 1 -- 172.21.15.49:0/545390860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08580ad4e0 con 0x7f08580ce9e0 2024-04-15T15:20:36.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.750+0000 7f085d400640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08581461c0 0x7f08581f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:36.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.750+0000 7f085ca00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0858072420 0x7f08581f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:36.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.750+0000 7f0857e00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08580ce9e0 0x7f08580aca20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f084400d830 tx=0x7f084400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:36.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.750+0000 7f0854c00640 1 -- 172.21.15.49:0/545390860 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084400b6c0 con 0x7f08580ce9e0 2024-04-15T15:20:36.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.750+0000 7f0854c00640 1 -- 172.21.15.49:0/545390860 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0844019070 con 0x7f08580ce9e0 2024-04-15T15:20:36.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.750+0000 7f0854c00640 1 -- 172.21.15.49:0/545390860 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084400ba80 con 0x7f08580ce9e0 2024-04-15T15:20:36.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.750+0000 7f085de00640 1 -- 172.21.15.49:0/545390860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08580ad7c0 con 0x7f08580ce9e0 2024-04-15T15:20:36.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.750+0000 7f085de00640 1 -- 172.21.15.49:0/545390860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08580add80 con 0x7f08580ce9e0 2024-04-15T15:20:36.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.752+0000 7f085de00640 1 -- 172.21.15.49:0/545390860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08580755c0 con 0x7f08580ce9e0 2024-04-15T15:20:36.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.753+0000 7f0854c00640 1 -- 172.21.15.49:0/545390860 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f084401e460 con 0x7f08580ce9e0 2024-04-15T15:20:36.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.754+0000 7f0854c00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f083806c9d0 0x7f083806ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:36.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.754+0000 7f0854c00640 1 -- 172.21.15.49:0/545390860 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f084408c250 con 0x7f08580ce9e0 2024-04-15T15:20:36.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.754+0000 7f085ca00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f083806c9d0 0x7f083806ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:36.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.755+0000 7f085ca00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f083806c9d0 0x7f083806ee90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f084c036000 tx=0x7f084c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:36.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.758+0000 7f0854c00640 1 -- 172.21.15.49:0/545390860 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f084405a740 con 0x7f08580ce9e0 2024-04-15T15:20:36.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.965+0000 7f085de00640 1 -- 172.21.15.49:0/545390860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0858072e40 con 0x7f08580ce9e0 2024-04-15T15:20:36.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.966+0000 7f0854c00640 1 -- 172.21.15.49:0/545390860 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f084405a040 con 0x7f08580ce9e0 2024-04-15T15:20:36.968 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:20:36.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.969+0000 7f085de00640 1 -- 172.21.15.49:0/545390860 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f083806c9d0 msgr2=0x7f083806ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.969+0000 7f085de00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f083806c9d0 0x7f083806ee90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f084c036000 tx=0x7f084c03c040 comp rx=0 tx=0).stop 2024-04-15T15:20:36.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.969+0000 7f085de00640 1 -- 172.21.15.49:0/545390860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08580ce9e0 msgr2=0x7f08580aca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:36.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.969+0000 7f085de00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08580ce9e0 0x7f08580aca20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f084400d830 tx=0x7f084400dcf0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.969+0000 7f085de00640 1 -- 172.21.15.49:0/545390860 shutdown_connections 2024-04-15T15:20:36.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.969+0000 7f085de00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f083806c9d0 0x7f083806ee90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.969+0000 7f085de00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08581461c0 0x7f08581f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.969+0000 7f085de00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08580ce9e0 0x7f08580aca20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.969+0000 7f085de00640 1 --2- 172.21.15.49:0/545390860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0858072420 0x7f08581f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:36.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.969+0000 7f085de00640 1 -- 172.21.15.49:0/545390860 >> 172.21.15.49:0/545390860 conn(0x7f085806b930 msgr2=0x7f08580b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:36.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.970+0000 7f085de00640 1 -- 172.21.15.49:0/545390860 shutdown_connections 2024-04-15T15:20:36.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:36.970+0000 7f085de00640 1 -- 172.21.15.49:0/545390860 wait complete. 2024-04-15T15:20:36.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.225+0000 7fd12a000640 1 Processor -- start 2024-04-15T15:20:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.225+0000 7fd12a000640 1 -- start start 2024-04-15T15:20:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.227+0000 7fd12a000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd12415b410 0x7fd12415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.227+0000 7fd12a000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd1240cf530 0x7fd1240cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.227+0000 7fd12a000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd12415cae0 con 0x7fd12415f110 2024-04-15T15:20:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.227+0000 7fd12a000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd1240cd360 con 0x7fd12415b410 2024-04-15T15:20:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.227+0000 7fd12a000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fd1240cd640 con 0x7fd1240cf530 2024-04-15T15:20:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.227+0000 7fd128c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd12415b410 0x7fd12415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.227+0000 7fd128c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd12415b410 0x7fd12415c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41526/0 (socket says 172.21.15.49:41526) 2024-04-15T15:20:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.227+0000 7fd128c00640 1 -- 172.21.15.49:0/3287928305 learned_addr learned my addr 172.21.15.49:0/3287928305 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:37.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.228+0000 7fd128c00640 1 -- 172.21.15.49:0/3287928305 >> v2:172.21.15.107:3300/0 conn(0x7fd1240cf530 msgr2=0x7fd1240cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:37.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.228+0000 7fd128c00640 1 --2- 172.21.15.49:0/3287928305 >> v2:172.21.15.107:3300/0 conn(0x7fd1240cf530 0x7fd1240cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.228+0000 7fd128c00640 1 -- 172.21.15.49:0/3287928305 >> v1:172.21.15.107:6789/0 conn(0x7fd12415f110 legacy=0x7fd12415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:37.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.228+0000 7fd128c00640 1 -- 172.21.15.49:0/3287928305 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1240cd920 con 0x7fd12415b410 2024-04-15T15:20:37.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.229+0000 7fd128c00640 1 --2- 172.21.15.49:0/3287928305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415b410 0x7fd12415c5a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd1240b3310 tx=0x7fd11802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c127701b03c78f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.229+0000 7fd123400640 1 -- 172.21.15.49:0/3287928305 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11802e070 con 0x7fd12415b410 2024-04-15T15:20:37.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.229+0000 7fd123400640 1 -- 172.21.15.49:0/3287928305 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd118037070 con 0x7fd12415b410 2024-04-15T15:20:37.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.229+0000 7fd123400640 1 -- 172.21.15.49:0/3287928305 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118031a00 con 0x7fd12415b410 2024-04-15T15:20:37.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.230+0000 7fd12a000640 1 -- 172.21.15.49:0/3287928305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415b410 msgr2=0x7fd12415c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.230+0000 7fd12a000640 1 --2- 172.21.15.49:0/3287928305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415b410 0x7fd12415c5a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd1240b3310 tx=0x7fd11802b880 comp rx=0 tx=0).stop 2024-04-15T15:20:37.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.230+0000 7fd12a000640 1 -- 172.21.15.49:0/3287928305 shutdown_connections 2024-04-15T15:20:37.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.230+0000 7fd12a000640 1 --2- 172.21.15.49:0/3287928305 >> v2:172.21.15.107:3300/0 conn(0x7fd1240cf530 0x7fd1240cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.230+0000 7fd12a000640 1 --2- 172.21.15.49:0/3287928305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415b410 0x7fd12415c5a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.230+0000 7fd12a000640 1 -- 172.21.15.49:0/3287928305 >> 172.21.15.49:0/3287928305 conn(0x7fd1240b23a0 msgr2=0x7fd1240b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.230+0000 7fd12a000640 1 -- 172.21.15.49:0/3287928305 shutdown_connections 2024-04-15T15:20:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.230+0000 7fd12a000640 1 -- 172.21.15.49:0/3287928305 wait complete. 2024-04-15T15:20:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.232+0000 7fd12a000640 1 Processor -- start 2024-04-15T15:20:37.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.232+0000 7fd12a000640 1 -- start start 2024-04-15T15:20:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.233+0000 7fd12a000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1240cf530 0x7fd12414d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.233+0000 7fd12a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12415b410 0x7fd12414b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.233+0000 7fd12a000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415f110 0x7fd124151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.233+0000 7fd12a000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240cfee0 con 0x7fd12415f110 2024-04-15T15:20:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.233+0000 7fd12a000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240cdee0 con 0x7fd12415b410 2024-04-15T15:20:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.233+0000 7fd12a000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd1240cd030 con 0x7fd1240cf530 2024-04-15T15:20:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd129600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415f110 0x7fd124151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd129600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415f110 0x7fd124151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41532/0 (socket says 172.21.15.49:41532) 2024-04-15T15:20:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd129600640 1 -- 172.21.15.49:0/4122341063 learned_addr learned my addr 172.21.15.49:0/4122341063 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd123e00640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12415b410 0x7fd12414b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd129600640 1 -- 172.21.15.49:0/4122341063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1240cf530 msgr2=0x7fd12414d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd128c00640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1240cf530 0x7fd12414d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd129600640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1240cf530 0x7fd12414d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd129600640 1 -- 172.21.15.49:0/4122341063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12415b410 msgr2=0x7fd12414b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd129600640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12415b410 0x7fd12414b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd129600640 1 -- 172.21.15.49:0/4122341063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12414d690 con 0x7fd12415f110 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd123e00640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12415b410 0x7fd12414b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd128c00640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1240cf530 0x7fd12414d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.234+0000 7fd129600640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415f110 0x7fd124151560 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00c7b0 tx=0x7fd11c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.235+0000 7fd120c00640 1 -- 172.21.15.49:0/4122341063 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c017070 con 0x7fd12415f110 2024-04-15T15:20:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.235+0000 7fd120c00640 1 -- 172.21.15.49:0/4122341063 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11c003da0 con 0x7fd12415f110 2024-04-15T15:20:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.235+0000 7fd120c00640 1 -- 172.21.15.49:0/4122341063 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00ce60 con 0x7fd12415f110 2024-04-15T15:20:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.235+0000 7fd12a000640 1 -- 172.21.15.49:0/4122341063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12414ef40 con 0x7fd12415f110 2024-04-15T15:20:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.235+0000 7fd12a000640 1 -- 172.21.15.49:0/4122341063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12414f500 con 0x7fd12415f110 2024-04-15T15:20:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.236+0000 7fd12a000640 1 -- 172.21.15.49:0/4122341063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1240755c0 con 0x7fd12415f110 2024-04-15T15:20:37.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.238+0000 7fd120c00640 1 -- 172.21.15.49:0/4122341063 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd11c0052b0 con 0x7fd12415f110 2024-04-15T15:20:37.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.238+0000 7fd120c00640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd10406c980 0x7fd10406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.239+0000 7fd120c00640 1 -- 172.21.15.49:0/4122341063 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fd11c08b0d0 con 0x7fd12415f110 2024-04-15T15:20:37.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.239+0000 7fd128c00640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd10406c980 0x7fd10406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.240+0000 7fd128c00640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd10406c980 0x7fd10406ee40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd118036000 tx=0x7fd118008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.243+0000 7fd120c00640 1 -- 172.21.15.49:0/4122341063 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd11c059540 con 0x7fd12415f110 2024-04-15T15:20:37.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.469+0000 7fd12a000640 1 -- 172.21.15.49:0/4122341063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1240b0c30 con 0x7fd12415f110 2024-04-15T15:20:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.470+0000 7fd120c00640 1 -- 172.21.15.49:0/4122341063 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) ==== 80+0+122 (secure 0 0 0) 0x7fd11c058e40 con 0x7fd12415f110 2024-04-15T15:20:37.472 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6837/1023024717 2024-04-16T15:20:28.618766+0000 2024-04-15T15:20:37.472 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6836/1023024717 2024-04-16T15:20:28.618766+0000 2024-04-15T15:20:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:listed 2 entries 2024-04-15T15:20:37.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.472+0000 7fd12a000640 1 -- 172.21.15.49:0/4122341063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd10406c980 msgr2=0x7fd10406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.472+0000 7fd12a000640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd10406c980 0x7fd10406ee40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd118036000 tx=0x7fd118008030 comp rx=0 tx=0).stop 2024-04-15T15:20:37.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.472+0000 7fd12a000640 1 -- 172.21.15.49:0/4122341063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415f110 msgr2=0x7fd124151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.472+0000 7fd12a000640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415f110 0x7fd124151560 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00c7b0 tx=0x7fd11c00cc70 comp rx=0 tx=0).stop 2024-04-15T15:20:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.472+0000 7fd12a000640 1 -- 172.21.15.49:0/4122341063 shutdown_connections 2024-04-15T15:20:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.472+0000 7fd12a000640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd10406c980 0x7fd10406ee40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.472+0000 7fd12a000640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12415f110 0x7fd124151560 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.472+0000 7fd12a000640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12415b410 0x7fd12414b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.472+0000 7fd12a000640 1 --2- 172.21.15.49:0/4122341063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1240cf530 0x7fd12414d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.472+0000 7fd12a000640 1 -- 172.21.15.49:0/4122341063 >> 172.21.15.49:0/4122341063 conn(0x7fd1240b23a0 msgr2=0x7fd1240b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.473+0000 7fd12a000640 1 -- 172.21.15.49:0/4122341063 shutdown_connections 2024-04-15T15:20:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.473+0000 7fd12a000640 1 -- 172.21.15.49:0/4122341063 wait complete. 2024-04-15T15:20:37.494 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-15T15:20:37.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:37.494 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-15T15:20:37.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:20:37.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:37.603 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-15T15:20:37.656 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:20:37.656 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:20:37.656 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:37.711 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:20:37.711 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:37.759 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:37.768 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:20:37.814 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:20:37.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.986+0000 7f951f200640 1 Processor -- start 2024-04-15T15:20:37.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.986+0000 7f951f200640 1 -- start start 2024-04-15T15:20:37.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.988+0000 7f951f200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9520148130 0x7f952015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.988+0000 7f951f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 0x7f9520071f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.988+0000 7f951f200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9520162940 con 0x7f9520072490 2024-04-15T15:20:37.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.988+0000 7f951f200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f9520148510 con 0x7f9520148130 2024-04-15T15:20:37.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.988+0000 7f951f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f952015fd70 con 0x7f9520162380 2024-04-15T15:20:37.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.989+0000 7f9515400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 0x7f9520071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.989+0000 7f9515400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 0x7f9520071f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:42536/0 (socket says 172.21.15.49:42536) 2024-04-15T15:20:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.989+0000 7f9515400640 1 -- 172.21.15.49:0/136589450 learned_addr learned my addr 172.21.15.49:0/136589450 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.989+0000 7f951de00640 1 --2- 172.21.15.49:0/136589450 >> v2:172.21.15.107:3300/0 conn(0x7f9520148130 0x7f952015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.990+0000 7f9515400640 1 -- 172.21.15.49:0/136589450 >> v2:172.21.15.107:3300/0 conn(0x7f9520148130 msgr2=0x7f952015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.990+0000 7f9515400640 1 --2- 172.21.15.49:0/136589450 >> v2:172.21.15.107:3300/0 conn(0x7f9520148130 0x7f952015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.990+0000 7f9515400640 1 -- 172.21.15.49:0/136589450 >> v1:172.21.15.107:6789/0 conn(0x7f9520072490 legacy=0x7f952006dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.990+0000 7f9515400640 1 -- 172.21.15.49:0/136589450 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9520162c20 con 0x7f9520162380 2024-04-15T15:20:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.991+0000 7f9515400640 1 --2- 172.21.15.49:0/136589450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 0x7f9520071f50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9508009870 tx=0x7f950802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a62c124a9899e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.991+0000 7f951d400640 1 -- 172.21.15.49:0/136589450 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508034070 con 0x7f9520162380 2024-04-15T15:20:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.992+0000 7f951d400640 1 -- 172.21.15.49:0/136589450 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9508030840 con 0x7f9520162380 2024-04-15T15:20:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.992+0000 7f951d400640 1 -- 172.21.15.49:0/136589450 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508030c10 con 0x7f9520162380 2024-04-15T15:20:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.992+0000 7f951f200640 1 -- 172.21.15.49:0/136589450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 msgr2=0x7f9520071f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.992+0000 7f951f200640 1 --2- 172.21.15.49:0/136589450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 0x7f9520071f50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9508009870 tx=0x7f950802b980 comp rx=0 tx=0).stop 2024-04-15T15:20:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.992+0000 7f951f200640 1 -- 172.21.15.49:0/136589450 shutdown_connections 2024-04-15T15:20:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.992+0000 7f951f200640 1 --2- 172.21.15.49:0/136589450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 0x7f9520071f50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.992+0000 7f951f200640 1 --2- 172.21.15.49:0/136589450 >> v2:172.21.15.107:3300/0 conn(0x7f9520148130 0x7f952015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.992+0000 7f951f200640 1 -- 172.21.15.49:0/136589450 >> 172.21.15.49:0/136589450 conn(0x7f952006b930 msgr2=0x7f952006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.992+0000 7f951f200640 1 -- 172.21.15.49:0/136589450 shutdown_connections 2024-04-15T15:20:37.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.992+0000 7f951f200640 1 -- 172.21.15.49:0/136589450 wait complete. 2024-04-15T15:20:37.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.994+0000 7f951f200640 1 Processor -- start 2024-04-15T15:20:37.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.994+0000 7f951f200640 1 -- start start 2024-04-15T15:20:37.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.995+0000 7f951f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9520072490 0x7f95200ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.995+0000 7f951f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9520148130 0x7f952013a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.995+0000 7f951f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 0x7f952013abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:37.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.995+0000 7f951f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f95200aeb90 con 0x7f9520148130 2024-04-15T15:20:37.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.995+0000 7f951f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95201631e0 con 0x7f9520072490 2024-04-15T15:20:37.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.995+0000 7f951f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f95200aed10 con 0x7f9520162380 2024-04-15T15:20:37.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.996+0000 7f9515400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9520148130 0x7f952013a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.996+0000 7f9515400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9520148130 0x7f952013a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41540/0 (socket says 172.21.15.49:41540) 2024-04-15T15:20:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.996+0000 7f9515400640 1 -- 172.21.15.49:0/1235328123 learned_addr learned my addr 172.21.15.49:0/1235328123 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.996+0000 7f951e800640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 0x7f952013abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.996+0000 7f951de00640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9520072490 0x7f95200ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.996+0000 7f9515400640 1 -- 172.21.15.49:0/1235328123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 msgr2=0x7f952013abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.996+0000 7f9515400640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 0x7f952013abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.996+0000 7f9515400640 1 -- 172.21.15.49:0/1235328123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9520072490 msgr2=0x7f95200ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.996+0000 7f9515400640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9520072490 0x7f95200ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.996+0000 7f9515400640 1 -- 172.21.15.49:0/1235328123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95200cef00 con 0x7f9520148130 2024-04-15T15:20:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.997+0000 7f9515400640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9520148130 0x7f952013a170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f95080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.997+0000 7f9516a00640 1 -- 172.21.15.49:0/1235328123 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508034040 con 0x7f9520148130 2024-04-15T15:20:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.997+0000 7f9516a00640 1 -- 172.21.15.49:0/1235328123 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9508039d90 con 0x7f9520148130 2024-04-15T15:20:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.997+0000 7f951f200640 1 -- 172.21.15.49:0/1235328123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95200cf1e0 con 0x7f9520148130 2024-04-15T15:20:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.997+0000 7f9516a00640 1 -- 172.21.15.49:0/1235328123 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508043630 con 0x7f9520148130 2024-04-15T15:20:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.997+0000 7f951f200640 1 -- 172.21.15.49:0/1235328123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95200af0b0 con 0x7f9520148130 2024-04-15T15:20:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.998+0000 7f951f200640 1 -- 172.21.15.49:0/1235328123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95200755c0 con 0x7f9520148130 2024-04-15T15:20:38.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:37.999+0000 7f9516a00640 1 -- 172.21.15.49:0/1235328123 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f950804a060 con 0x7f9520148130 2024-04-15T15:20:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.000+0000 7f9516a00640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f94fc06c980 0x7f94fc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.000+0000 7f9516a00640 1 -- 172.21.15.49:0/1235328123 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f95080adbf0 con 0x7f9520148130 2024-04-15T15:20:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.001+0000 7f951de00640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f94fc06c980 0x7f94fc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.001+0000 7f951de00640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f94fc06c980 0x7f94fc06ee40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9510009ad0 tx=0x7f9510008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.005+0000 7f9516a00640 1 -- 172.21.15.49:0/1235328123 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f950803e070 con 0x7f9520148130 2024-04-15T15:20:38.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.212+0000 7f951f200640 1 -- 172.21.15.49:0/1235328123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f952006f060 con 0x7f9520148130 2024-04-15T15:20:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.213+0000 7f9516a00640 1 -- 172.21.15.49:0/1235328123 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16798 (secure 0 0 0) 0x7f9508037e40 con 0x7f9520148130 2024-04-15T15:20:38.216 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:38.216 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:28.618835+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6837/1023024717":"2024-04-16T15:20:28.618766+0000","172.21.15.49:6836/1023024717":"2024-04-16T15:20:28.618766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 -- 172.21.15.49:0/1235328123 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f94fc06c980 msgr2=0x7f94fc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f94fc06c980 0x7f94fc06ee40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9510009ad0 tx=0x7f9510008040 comp rx=0 tx=0).stop 2024-04-15T15:20:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 -- 172.21.15.49:0/1235328123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9520148130 msgr2=0x7f952013a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9520148130 0x7f952013a170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f95080046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 -- 172.21.15.49:0/1235328123 shutdown_connections 2024-04-15T15:20:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f94fc06c980 0x7f94fc06ee40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9520162380 0x7f952013abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9520148130 0x7f952013a170 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 --2- 172.21.15.49:0/1235328123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9520072490 0x7f95200ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 -- 172.21.15.49:0/1235328123 >> 172.21.15.49:0/1235328123 conn(0x7f952006b930 msgr2=0x7f95200b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 -- 172.21.15.49:0/1235328123 shutdown_connections 2024-04-15T15:20:38.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.216+0000 7f951f200640 1 -- 172.21.15.49:0/1235328123 wait complete. 2024-04-15T15:20:38.238 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:58.063498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:59.009342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:38.239 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:38.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.471+0000 7f5eca400640 1 Processor -- start 2024-04-15T15:20:38.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.472+0000 7f5eca400640 1 -- start start 2024-04-15T15:20:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.473+0000 7f5eca400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5ec406ffb0 0x7f5ec415f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.473+0000 7f5eca400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5ec4159390 0x7f5ec415f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.473+0000 7f5eca400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec40b4670 con 0x7f5ec4158fb0 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.473+0000 7f5eca400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5ec40b5c90 con 0x7f5ec406ffb0 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.473+0000 7f5eca400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f5ec40b38b0 con 0x7f5ec4159390 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.473+0000 7f5ec3e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5ec406ffb0 0x7f5ec415f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.474+0000 7f5ec3e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5ec406ffb0 0x7f5ec415f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41554/0 (socket says 172.21.15.49:41554) 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.474+0000 7f5ec3e00640 1 -- 172.21.15.49:0/1694782097 learned_addr learned my addr 172.21.15.49:0/1694782097 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.474+0000 7f5ec3e00640 1 -- 172.21.15.49:0/1694782097 >> v2:172.21.15.107:3300/0 conn(0x7f5ec4159390 msgr2=0x7f5ec415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.474+0000 7f5ec3e00640 1 --2- 172.21.15.49:0/1694782097 >> v2:172.21.15.107:3300/0 conn(0x7f5ec4159390 0x7f5ec415f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.474+0000 7f5ec3e00640 1 -- 172.21.15.49:0/1694782097 >> v1:172.21.15.107:6789/0 conn(0x7f5ec4158fb0 legacy=0x7f5ec406f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.474+0000 7f5ec3e00640 1 -- 172.21.15.49:0/1694782097 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec40b3b90 con 0x7f5ec406ffb0 2024-04-15T15:20:38.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.475+0000 7f5ec3e00640 1 --2- 172.21.15.49:0/1694782097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 0x7f5ec415f440 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40b4150 tx=0x7f5eb002e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e529ef64b4b853 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5ec3400640 1 -- 172.21.15.49:0/1694782097 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb003b070 con 0x7f5ec406ffb0 2024-04-15T15:20:38.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5ec3400640 1 -- 172.21.15.49:0/1694782097 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb002ee10 con 0x7f5ec406ffb0 2024-04-15T15:20:38.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5ec3400640 1 -- 172.21.15.49:0/1694782097 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0036660 con 0x7f5ec406ffb0 2024-04-15T15:20:38.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5eca400640 1 -- 172.21.15.49:0/1694782097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 msgr2=0x7f5ec415f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5eca400640 1 --2- 172.21.15.49:0/1694782097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 0x7f5ec415f440 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40b4150 tx=0x7f5eb002e060 comp rx=0 tx=0).stop 2024-04-15T15:20:38.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5eca400640 1 -- 172.21.15.49:0/1694782097 shutdown_connections 2024-04-15T15:20:38.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5eca400640 1 --2- 172.21.15.49:0/1694782097 >> v2:172.21.15.107:3300/0 conn(0x7f5ec4159390 0x7f5ec415f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5eca400640 1 --2- 172.21.15.49:0/1694782097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 0x7f5ec415f440 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5eca400640 1 -- 172.21.15.49:0/1694782097 >> 172.21.15.49:0/1694782097 conn(0x7f5ec40b23a0 msgr2=0x7f5ec40b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5eca400640 1 -- 172.21.15.49:0/1694782097 shutdown_connections 2024-04-15T15:20:38.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.476+0000 7f5eca400640 1 -- 172.21.15.49:0/1694782097 wait complete. 2024-04-15T15:20:38.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.478+0000 7f5eca400640 1 Processor -- start 2024-04-15T15:20:38.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.478+0000 7f5eca400640 1 -- start start 2024-04-15T15:20:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.479+0000 7f5eca400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 0x7f5ec41481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.479+0000 7f5eca400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ec4158fb0 0x7f5ec41467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.479+0000 7f5eca400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec4159390 0x7f5ec4148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.479+0000 7f5eca400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec40b4c80 con 0x7f5ec406ffb0 2024-04-15T15:20:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.479+0000 7f5eca400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec415fec0 con 0x7f5ec4159390 2024-04-15T15:20:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.479+0000 7f5eca400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec40b4e00 con 0x7f5ec4158fb0 2024-04-15T15:20:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec9000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 0x7f5ec41481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec9000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 0x7f5ec41481f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41564/0 (socket says 172.21.15.49:41564) 2024-04-15T15:20:38.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec9000640 1 -- 172.21.15.49:0/1880303495 learned_addr learned my addr 172.21.15.49:0/1880303495 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:38.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec3e00640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ec4158fb0 0x7f5ec41467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec9a00640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec4159390 0x7f5ec4148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec9000640 1 -- 172.21.15.49:0/1880303495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ec4158fb0 msgr2=0x7f5ec41467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec9000640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ec4158fb0 0x7f5ec41467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec9000640 1 -- 172.21.15.49:0/1880303495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec4159390 msgr2=0x7f5ec4148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec9000640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec4159390 0x7f5ec4148ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec9000640 1 -- 172.21.15.49:0/1880303495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec4147150 con 0x7f5ec406ffb0 2024-04-15T15:20:38.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.480+0000 7f5ec9a00640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec4159390 0x7f5ec4148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:38.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.481+0000 7f5ec9000640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 0x7f5ec41481f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c440 tx=0x7f5eb800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.481+0000 7f5ec0c00640 1 -- 172.21.15.49:0/1880303495 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8017070 con 0x7f5ec406ffb0 2024-04-15T15:20:38.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.481+0000 7f5ec0c00640 1 -- 172.21.15.49:0/1880303495 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb80035f0 con 0x7f5ec406ffb0 2024-04-15T15:20:38.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.481+0000 7f5ec0c00640 1 -- 172.21.15.49:0/1880303495 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80039e0 con 0x7f5ec406ffb0 2024-04-15T15:20:38.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.481+0000 7f5eca400640 1 -- 172.21.15.49:0/1880303495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec4147430 con 0x7f5ec406ffb0 2024-04-15T15:20:38.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.481+0000 7f5eca400640 1 -- 172.21.15.49:0/1880303495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec41479f0 con 0x7f5ec406ffb0 2024-04-15T15:20:38.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.482+0000 7f5eca400640 1 -- 172.21.15.49:0/1880303495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec40755c0 con 0x7f5ec406ffb0 2024-04-15T15:20:38.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.483+0000 7f5ec0c00640 1 -- 172.21.15.49:0/1880303495 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5eb801a070 con 0x7f5ec406ffb0 2024-04-15T15:20:38.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.484+0000 7f5ec0c00640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5ea406c9b0 0x7f5ea406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.484+0000 7f5ec0c00640 1 -- 172.21.15.49:0/1880303495 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f5eb808af30 con 0x7f5ec406ffb0 2024-04-15T15:20:38.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.484+0000 7f5ec3e00640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5ea406c9b0 0x7f5ea406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.485+0000 7f5ec3e00640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5ea406c9b0 0x7f5ea406ee70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0038000 tx=0x7f5eb0032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.491+0000 7f5ec0c00640 1 -- 172.21.15.49:0/1880303495 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5eb80593a0 con 0x7f5ec406ffb0 2024-04-15T15:20:38.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.693+0000 7f5eca400640 1 -- 172.21.15.49:0/1880303495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ec406e400 con 0x7f5ec406ffb0 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.694+0000 7f5ec0c00640 1 -- 172.21.15.49:0/1880303495 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f5eb80010c0 con 0x7f5ec406ffb0 2024-04-15T15:20:38.696 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:20:38.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.697+0000 7f5eca400640 1 -- 172.21.15.49:0/1880303495 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5ea406c9b0 msgr2=0x7f5ea406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.697+0000 7f5eca400640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5ea406c9b0 0x7f5ea406ee70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0038000 tx=0x7f5eb0032000 comp rx=0 tx=0).stop 2024-04-15T15:20:38.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.697+0000 7f5eca400640 1 -- 172.21.15.49:0/1880303495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 msgr2=0x7f5ec41481f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.697+0000 7f5eca400640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 0x7f5ec41481f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c440 tx=0x7f5eb800c900 comp rx=0 tx=0).stop 2024-04-15T15:20:38.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.697+0000 7f5eca400640 1 -- 172.21.15.49:0/1880303495 shutdown_connections 2024-04-15T15:20:38.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.697+0000 7f5eca400640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5ea406c9b0 0x7f5ea406ee70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.697+0000 7f5eca400640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ec4158fb0 0x7f5ec41467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.697+0000 7f5eca400640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec4159390 0x7f5ec4148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.697+0000 7f5eca400640 1 --2- 172.21.15.49:0/1880303495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ec406ffb0 0x7f5ec41481f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.697+0000 7f5eca400640 1 -- 172.21.15.49:0/1880303495 >> 172.21.15.49:0/1880303495 conn(0x7f5ec40b23a0 msgr2=0x7f5ec415cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.698+0000 7f5eca400640 1 -- 172.21.15.49:0/1880303495 shutdown_connections 2024-04-15T15:20:38.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.698+0000 7f5eca400640 1 -- 172.21.15.49:0/1880303495 wait complete. 2024-04-15T15:20:38.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:38.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.943+0000 7f85b5200640 1 Processor -- start 2024-04-15T15:20:38.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.943+0000 7f85b5200640 1 -- start start 2024-04-15T15:20:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.945+0000 7f85b5200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f85b0147c90 0x7f85b006f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.945+0000 7f85b5200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 0x7f85b01626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.945+0000 7f85b5200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b00aebd0 con 0x7f85b01478b0 2024-04-15T15:20:38.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.945+0000 7f85b5200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f85b006fbb0 con 0x7f85b0147c90 2024-04-15T15:20:38.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.945+0000 7f85b5200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f85b00aee50 con 0x7f85b006f750 2024-04-15T15:20:38.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.946+0000 7f85afe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 0x7f85b01626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.946+0000 7f85afe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 0x7f85b01626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:42568/0 (socket says 172.21.15.49:42568) 2024-04-15T15:20:38.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.946+0000 7f85afe00640 1 -- 172.21.15.49:0/2085446884 learned_addr learned my addr 172.21.15.49:0/2085446884 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:38.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.946+0000 7f85aea00640 1 --2- 172.21.15.49:0/2085446884 >> v2:172.21.15.107:3300/0 conn(0x7f85b0147c90 0x7f85b006f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.947+0000 7f85afe00640 1 -- 172.21.15.49:0/2085446884 >> v2:172.21.15.107:3300/0 conn(0x7f85b0147c90 msgr2=0x7f85b006f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:38.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.947+0000 7f85afe00640 1 --2- 172.21.15.49:0/2085446884 >> v2:172.21.15.107:3300/0 conn(0x7f85b0147c90 0x7f85b006f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.947+0000 7f85afe00640 1 -- 172.21.15.49:0/2085446884 >> v1:172.21.15.107:6789/0 conn(0x7f85b01478b0 legacy=0x7f85b00afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.947+0000 7f85afe00640 1 -- 172.21.15.49:0/2085446884 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b00af110 con 0x7f85b006f750 2024-04-15T15:20:38.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.948+0000 7f85afe00640 1 --2- 172.21.15.49:0/2085446884 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 0x7f85b01626c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009e50 tx=0x7f85a802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28de8629aefab1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.949+0000 7f85ae000640 1 -- 172.21.15.49:0/2085446884 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a802cb00 con 0x7f85b006f750 2024-04-15T15:20:38.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.949+0000 7f85ae000640 1 -- 172.21.15.49:0/2085446884 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a802cca0 con 0x7f85b006f750 2024-04-15T15:20:38.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.949+0000 7f85ae000640 1 -- 172.21.15.49:0/2085446884 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8034610 con 0x7f85b006f750 2024-04-15T15:20:38.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.949+0000 7f85b5200640 1 -- 172.21.15.49:0/2085446884 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 msgr2=0x7f85b01626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.949+0000 7f85b5200640 1 --2- 172.21.15.49:0/2085446884 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 0x7f85b01626c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009e50 tx=0x7f85a802bf60 comp rx=0 tx=0).stop 2024-04-15T15:20:38.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.949+0000 7f85b5200640 1 -- 172.21.15.49:0/2085446884 shutdown_connections 2024-04-15T15:20:38.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.949+0000 7f85b5200640 1 --2- 172.21.15.49:0/2085446884 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 0x7f85b01626c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.949+0000 7f85b5200640 1 --2- 172.21.15.49:0/2085446884 >> v2:172.21.15.107:3300/0 conn(0x7f85b0147c90 0x7f85b006f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.949+0000 7f85b5200640 1 -- 172.21.15.49:0/2085446884 >> 172.21.15.49:0/2085446884 conn(0x7f85b006b930 msgr2=0x7f85b006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:38.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.950+0000 7f85b5200640 1 -- 172.21.15.49:0/2085446884 shutdown_connections 2024-04-15T15:20:38.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.950+0000 7f85b5200640 1 -- 172.21.15.49:0/2085446884 wait complete. 2024-04-15T15:20:38.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.951+0000 7f85b5200640 1 Processor -- start 2024-04-15T15:20:38.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.952+0000 7f85b5200640 1 -- start start 2024-04-15T15:20:38.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.952+0000 7f85b5200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 0x7f85b015e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.952+0000 7f85b5200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b01478b0 0x7f85b015db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.952+0000 7f85b5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85b0147c90 0x7f85b013d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.952+0000 7f85b5200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0162c00 con 0x7f85b01478b0 2024-04-15T15:20:38.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.952+0000 7f85b5200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85b00af6d0 con 0x7f85b0147c90 2024-04-15T15:20:38.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.952+0000 7f85b5200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f85b0162d80 con 0x7f85b006f750 2024-04-15T15:20:38.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.953+0000 7f85aea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b01478b0 0x7f85b015db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.953+0000 7f85aea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b01478b0 0x7f85b015db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41572/0 (socket says 172.21.15.49:41572) 2024-04-15T15:20:38.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.953+0000 7f85aea00640 1 -- 172.21.15.49:0/941955300 learned_addr learned my addr 172.21.15.49:0/941955300 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:38.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.953+0000 7f85af400640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 0x7f85b015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.953+0000 7f85afe00640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85b0147c90 0x7f85b013d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.953+0000 7f85aea00640 1 -- 172.21.15.49:0/941955300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 msgr2=0x7f85b015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.953+0000 7f85aea00640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 0x7f85b015e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.953+0000 7f85aea00640 1 -- 172.21.15.49:0/941955300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85b0147c90 msgr2=0x7f85b013d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:38.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.953+0000 7f85aea00640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85b0147c90 0x7f85b013d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:38.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.953+0000 7f85aea00640 1 -- 172.21.15.49:0/941955300 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b013e130 con 0x7f85b01478b0 2024-04-15T15:20:38.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.954+0000 7f85aea00640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b01478b0 0x7f85b015db60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f859c00e870 tx=0x7f859c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.954+0000 7f85a7400640 1 -- 172.21.15.49:0/941955300 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c017070 con 0x7f85b01478b0 2024-04-15T15:20:38.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.954+0000 7f85a7400640 1 -- 172.21.15.49:0/941955300 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c00c940 con 0x7f85b01478b0 2024-04-15T15:20:38.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.954+0000 7f85a7400640 1 -- 172.21.15.49:0/941955300 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c00cd30 con 0x7f85b01478b0 2024-04-15T15:20:38.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.954+0000 7f85b5200640 1 -- 172.21.15.49:0/941955300 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b013e410 con 0x7f85b01478b0 2024-04-15T15:20:38.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.954+0000 7f85b5200640 1 -- 172.21.15.49:0/941955300 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b013e9d0 con 0x7f85b01478b0 2024-04-15T15:20:38.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.955+0000 7f85b5200640 1 -- 172.21.15.49:0/941955300 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b00755c0 con 0x7f85b01478b0 2024-04-15T15:20:38.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.957+0000 7f85a7400640 1 -- 172.21.15.49:0/941955300 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f859c01a070 con 0x7f85b01478b0 2024-04-15T15:20:38.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.958+0000 7f85a7400640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f859006c980 0x7f859006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:38.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.958+0000 7f85a7400640 1 -- 172.21.15.49:0/941955300 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f859c08bfa0 con 0x7f85b01478b0 2024-04-15T15:20:38.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.958+0000 7f85af400640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f859006c980 0x7f859006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:38.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.959+0000 7f85af400640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f859006c980 0x7f859006ee40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f85a0000f60 tx=0x7f85a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:38.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:38.963+0000 7f85a7400640 1 -- 172.21.15.49:0/941955300 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f859c05a1f0 con 0x7f85b01478b0 2024-04-15T15:20:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.168+0000 7f85b5200640 1 -- 172.21.15.49:0/941955300 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f85b00b1760 con 0x7f85b01478b0 2024-04-15T15:20:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.169+0000 7f85a7400640 1 -- 172.21.15.49:0/941955300 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7f859c059af0 con 0x7f85b01478b0 2024-04-15T15:20:39.171 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:20:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.172+0000 7f85b5200640 1 -- 172.21.15.49:0/941955300 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f859006c980 msgr2=0x7f859006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.172+0000 7f85b5200640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f859006c980 0x7f859006ee40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f85a0000f60 tx=0x7f85a0009040 comp rx=0 tx=0).stop 2024-04-15T15:20:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.172+0000 7f85b5200640 1 -- 172.21.15.49:0/941955300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b01478b0 msgr2=0x7f85b015db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.172+0000 7f85b5200640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b01478b0 0x7f85b015db60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f859c00e870 tx=0x7f859c00ed30 comp rx=0 tx=0).stop 2024-04-15T15:20:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.172+0000 7f85b5200640 1 -- 172.21.15.49:0/941955300 shutdown_connections 2024-04-15T15:20:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.172+0000 7f85b5200640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f859006c980 0x7f859006ee40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.172+0000 7f85b5200640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85b0147c90 0x7f85b013d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.172+0000 7f85b5200640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b01478b0 0x7f85b015db60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.172+0000 7f85b5200640 1 --2- 172.21.15.49:0/941955300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85b006f750 0x7f85b015e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.172+0000 7f85b5200640 1 -- 172.21.15.49:0/941955300 >> 172.21.15.49:0/941955300 conn(0x7f85b006b930 msgr2=0x7f85b00b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.173+0000 7f85b5200640 1 -- 172.21.15.49:0/941955300 shutdown_connections 2024-04-15T15:20:39.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.173+0000 7f85b5200640 1 -- 172.21.15.49:0/941955300 wait complete. 2024-04-15T15:20:39.192 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:39.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.420+0000 7f70cda00640 1 Processor -- start 2024-04-15T15:20:39.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.420+0000 7f70cda00640 1 -- start start 2024-04-15T15:20:39.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.421+0000 7f70cda00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f70c80b54b0 0x7f70c80b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.422+0000 7f70cda00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c80b4410 con 0x7f70c8145170 2024-04-15T15:20:39.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.422+0000 7f70cda00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c80b46f0 con 0x7f70c806f260 2024-04-15T15:20:39.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.422+0000 7f70cda00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f70c8145550 con 0x7f70c80b54b0 2024-04-15T15:20:39.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.422+0000 7f70c6000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f70c80b54b0 0x7f70c80b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.422+0000 7f70c6000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f70c80b54b0 0x7f70c80b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41578/0 (socket says 172.21.15.49:41578) 2024-04-15T15:20:39.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.422+0000 7f70c6000640 1 -- 172.21.15.49:0/1880240262 learned_addr learned my addr 172.21.15.49:0/1880240262 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:39.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.422+0000 7f70c6000640 1 -- 172.21.15.49:0/1880240262 >> v1:172.21.15.107:6789/0 conn(0x7f70c806f260 legacy=0x7f70c806f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:39.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.422+0000 7f70c6000640 1 -- 172.21.15.49:0/1880240262 >> v1:172.21.15.49:6789/0 conn(0x7f70c8145170 legacy=0x7f70c80cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.422+0000 7f70c6000640 1 -- 172.21.15.49:0/1880240262 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c80b49d0 con 0x7f70c80b54b0 2024-04-15T15:20:39.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.424+0000 7f70c6000640 1 --2- 172.21.15.49:0/1880240262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c80b54b0 0x7f70c80b5940 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f70c81456d0 tx=0x7f70b402e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec088dc09b1dd71 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:39.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.424+0000 7f70c5600640 1 -- 172.21.15.49:0/1880240262 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b400b0d0 con 0x7f70c80b54b0 2024-04-15T15:20:39.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.424+0000 7f70c5600640 1 -- 172.21.15.49:0/1880240262 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b400b270 con 0x7f70c80b54b0 2024-04-15T15:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.424+0000 7f70c5600640 1 -- 172.21.15.49:0/1880240262 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4009600 con 0x7f70c80b54b0 2024-04-15T15:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.424+0000 7f70cda00640 1 -- 172.21.15.49:0/1880240262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c80b54b0 msgr2=0x7f70c80b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.425+0000 7f70cda00640 1 --2- 172.21.15.49:0/1880240262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c80b54b0 0x7f70c80b5940 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f70c81456d0 tx=0x7f70b402e160 comp rx=0 tx=0).stop 2024-04-15T15:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.425+0000 7f70cda00640 1 -- 172.21.15.49:0/1880240262 shutdown_connections 2024-04-15T15:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.425+0000 7f70cda00640 1 --2- 172.21.15.49:0/1880240262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c80b54b0 0x7f70c80b5940 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.425+0000 7f70cda00640 1 -- 172.21.15.49:0/1880240262 >> 172.21.15.49:0/1880240262 conn(0x7f70c80b2460 msgr2=0x7f70c80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.425+0000 7f70cda00640 1 -- 172.21.15.49:0/1880240262 shutdown_connections 2024-04-15T15:20:39.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.425+0000 7f70cda00640 1 -- 172.21.15.49:0/1880240262 wait complete. 2024-04-15T15:20:39.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.427+0000 7f70cda00640 1 Processor -- start 2024-04-15T15:20:39.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.427+0000 7f70cda00640 1 -- start start 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70cda00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c806f260 0x7f70c81e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70cda00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c80b54b0 0x7f70c81f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70cda00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70c8145170 0x7f70c81e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70cda00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f70c80ce240 con 0x7f70c806f260 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70cda00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f70c80b4f90 con 0x7f70c80b54b0 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70cda00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f70c80b4030 con 0x7f70c8145170 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70c6a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c806f260 0x7f70c81e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70c6a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c806f260 0x7f70c81e3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41586/0 (socket says 172.21.15.49:41586) 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70c6a00640 1 -- 172.21.15.49:0/941789272 learned_addr learned my addr 172.21.15.49:0/941789272 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70c6000640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c80b54b0 0x7f70c81f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.428+0000 7f70c7400640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70c8145170 0x7f70c81e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.429+0000 7f70c6a00640 1 -- 172.21.15.49:0/941789272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70c8145170 msgr2=0x7f70c81e49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.429+0000 7f70c6a00640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70c8145170 0x7f70c81e49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.429+0000 7f70c6a00640 1 -- 172.21.15.49:0/941789272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c80b54b0 msgr2=0x7f70c81f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.429+0000 7f70c6a00640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c80b54b0 0x7f70c81f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.429+0000 7f70c6a00640 1 -- 172.21.15.49:0/941789272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c81f2d50 con 0x7f70c806f260 2024-04-15T15:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.429+0000 7f70c6a00640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c806f260 0x7f70c81e3d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c3c0 tx=0x7f70b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.429+0000 7f70bea00640 1 -- 172.21.15.49:0/941789272 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b80052d0 con 0x7f70c806f260 2024-04-15T15:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.429+0000 7f70bea00640 1 -- 172.21.15.49:0/941789272 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b8005490 con 0x7f70c806f260 2024-04-15T15:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.429+0000 7f70bea00640 1 -- 172.21.15.49:0/941789272 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8007260 con 0x7f70c806f260 2024-04-15T15:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.429+0000 7f70cda00640 1 -- 172.21.15.49:0/941789272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c81f3030 con 0x7f70c806f260 2024-04-15T15:20:39.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.430+0000 7f70cda00640 1 -- 172.21.15.49:0/941789272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c81f35f0 con 0x7f70c806f260 2024-04-15T15:20:39.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.431+0000 7f70cda00640 1 -- 172.21.15.49:0/941789272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c8068c80 con 0x7f70c806f260 2024-04-15T15:20:39.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.432+0000 7f70bea00640 1 -- 172.21.15.49:0/941789272 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f70b8007400 con 0x7f70c806f260 2024-04-15T15:20:39.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.433+0000 7f70bea00640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f70a806c980 0x7f70a806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.433+0000 7f70bea00640 1 -- 172.21.15.49:0/941789272 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f70b808b150 con 0x7f70c806f260 2024-04-15T15:20:39.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.433+0000 7f70c6000640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f70a806c980 0x7f70a806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.434+0000 7f70c6000640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f70a806c980 0x7f70a806ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f70b4037000 tx=0x7f70b4032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:39.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.438+0000 7f70bea00640 1 -- 172.21.15.49:0/941789272 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70b80594a0 con 0x7f70c806f260 2024-04-15T15:20:39.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.653+0000 7f70cda00640 1 -- 172.21.15.49:0/941789272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f70c80b5b50 con 0x7f70c806f260 2024-04-15T15:20:39.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.654+0000 7f70bea00640 1 -- 172.21.15.49:0/941789272 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f70b8058da0 con 0x7f70c806f260 2024-04-15T15:20:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:20:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.657+0000 7f70cda00640 1 -- 172.21.15.49:0/941789272 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f70a806c980 msgr2=0x7f70a806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.657+0000 7f70cda00640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f70a806c980 0x7f70a806ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f70b4037000 tx=0x7f70b4032000 comp rx=0 tx=0).stop 2024-04-15T15:20:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.657+0000 7f70cda00640 1 -- 172.21.15.49:0/941789272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c806f260 msgr2=0x7f70c81e3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.657+0000 7f70cda00640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c806f260 0x7f70c81e3d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c3c0 tx=0x7f70b800c880 comp rx=0 tx=0).stop 2024-04-15T15:20:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.658+0000 7f70cda00640 1 -- 172.21.15.49:0/941789272 shutdown_connections 2024-04-15T15:20:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.658+0000 7f70cda00640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f70a806c980 0x7f70a806ee40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.658+0000 7f70cda00640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70c8145170 0x7f70c81e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.658+0000 7f70cda00640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c80b54b0 0x7f70c81f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.658+0000 7f70cda00640 1 --2- 172.21.15.49:0/941789272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c806f260 0x7f70c81e3d40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.658+0000 7f70cda00640 1 -- 172.21.15.49:0/941789272 >> 172.21.15.49:0/941789272 conn(0x7f70c80b2460 msgr2=0x7f70c80b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:39.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.658+0000 7f70cda00640 1 -- 172.21.15.49:0/941789272 shutdown_connections 2024-04-15T15:20:39.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.658+0000 7f70cda00640 1 -- 172.21.15.49:0/941789272 wait complete. 2024-04-15T15:20:39.679 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:39.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.903+0000 7f5b1e800640 1 Processor -- start 2024-04-15T15:20:39.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.903+0000 7f5b1e800640 1 -- start start 2024-04-15T15:20:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.905+0000 7f5b1e800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b180ae790 0x7f5b18162420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.905+0000 7f5b1e800640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b180b3e90 con 0x7f5b18148050 2024-04-15T15:20:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.905+0000 7f5b1e800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b180afba0 con 0x7f5b18162960 2024-04-15T15:20:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.905+0000 7f5b1e800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f5b180aed00 con 0x7f5b180ae790 2024-04-15T15:20:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.905+0000 7f5b1d400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5b18148050 0x7f5b180b3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60120/0 (socket says 172.21.15.49:60120) 2024-04-15T15:20:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.905+0000 7f5b1d400640 1 -- 172.21.15.49:0/1561031121 learned_addr learned my addr 172.21.15.49:0/1561031121 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.906+0000 7f5b1ca00640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b180ae790 0x7f5b18162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.906+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317533402 0 0) 0x7f5b180b3e90 con 0x7f5b18148050 2024-04-15T15:20:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.906+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b00003520 con 0x7f5b18148050 2024-04-15T15:20:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.906+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 344498310 0 0) 0x7f5b00003520 con 0x7f5b18148050 2024-04-15T15:20:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.906+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b180ae790 msgr2=0x7f5b18162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.906+0000 7f5b17e00640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b180ae790 0x7f5b18162420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.906+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 >> v1:172.21.15.107:6789/0 conn(0x7f5b18162960 legacy=0x7f5b180b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.906+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b180aefe0 con 0x7f5b18148050 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.906+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f5b0c001670 con 0x7f5b18148050 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 >> v1:172.21.15.49:6789/0 conn(0x7f5b18148050 legacy=0x7f5b180b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b17e00640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b180ae790 0x7f5b000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b17e00640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b00003f20 0x7f5b00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b17e00640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b00004d40 0x7f5b000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180aed00 con 0x7f5b00004d40 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0000ba70 con 0x7f5b00003f20 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0000bc70 con 0x7f5b180ae790 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b1de00640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b00004d40 0x7f5b000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b1ca00640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b00003f20 0x7f5b00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b1d400640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b180ae790 0x7f5b000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b1de00640 1 -- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b180ae790 msgr2=0x7f5b000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b1de00640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b180ae790 0x7f5b000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b1de00640 1 -- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b00003f20 msgr2=0x7f5b00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b1de00640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b00003f20 0x7f5b00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.907+0000 7f5b1de00640 1 -- 172.21.15.49:0/1561031121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b00003520 con 0x7f5b00004d40 2024-04-15T15:20:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.909+0000 7f5b1de00640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b00004d40 0x7f5b000051c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5b10001f60 tx=0x7f5b00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80507eb44859d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.909+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10001b30 con 0x7f5b00004d40 2024-04-15T15:20:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.909+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b1002d710 con 0x7f5b00004d40 2024-04-15T15:20:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.909+0000 7f5b17e00640 1 -- 172.21.15.49:0/1561031121 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1002db00 con 0x7f5b00004d40 2024-04-15T15:20:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.910+0000 7f5b1e800640 1 -- 172.21.15.49:0/1561031121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b00004d40 msgr2=0x7f5b000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.910+0000 7f5b1e800640 1 --2- 172.21.15.49:0/1561031121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b00004d40 0x7f5b000051c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5b10001f60 tx=0x7f5b00005830 comp rx=0 tx=0).stop 2024-04-15T15:20:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.910+0000 7f5b1de00640 1 -- 172.21.15.49:0/1561031121 reap_dead start 2024-04-15T15:20:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.910+0000 7f5b1e800640 1 -- 172.21.15.49:0/1561031121 shutdown_connections 2024-04-15T15:20:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.910+0000 7f5b1e800640 1 -- 172.21.15.49:0/1561031121 >> 172.21.15.49:0/1561031121 conn(0x7f5b180b23a0 msgr2=0x7f5b180b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.910+0000 7f5b1e800640 1 -- 172.21.15.49:0/1561031121 shutdown_connections 2024-04-15T15:20:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.910+0000 7f5b1e800640 1 -- 172.21.15.49:0/1561031121 wait complete. 2024-04-15T15:20:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.912+0000 7f5b1e800640 1 Processor -- start 2024-04-15T15:20:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.912+0000 7f5b1e800640 1 -- start start 2024-04-15T15:20:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1e800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b00004d40 0x7f5b181540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b1815e2c0 0x7f5b18161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1e800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1813bfe0 0x7f5b181425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1e800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18148430 con 0x7f5b1813bfe0 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1e800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180afe40 con 0x7f5b1815e2c0 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1e800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f5b181485b0 con 0x7f5b00004d40 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1813bfe0 0x7f5b181425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1813bfe0 0x7f5b181425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41596/0 (socket says 172.21.15.49:41596) 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1de00640 1 -- 172.21.15.49:0/3549765203 learned_addr learned my addr 172.21.15.49:0/3549765203 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1ca00640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b1815e2c0 0x7f5b18161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1d400640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b00004d40 0x7f5b181540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1de00640 1 -- 172.21.15.49:0/3549765203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b00004d40 msgr2=0x7f5b181540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.913+0000 7f5b1de00640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b00004d40 0x7f5b181540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.914+0000 7f5b1de00640 1 -- 172.21.15.49:0/3549765203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b1815e2c0 msgr2=0x7f5b18161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.914+0000 7f5b1de00640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b1815e2c0 0x7f5b18161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.914+0000 7f5b1de00640 1 -- 172.21.15.49:0/3549765203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b181615c0 con 0x7f5b1813bfe0 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.914+0000 7f5b1d400640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b00004d40 0x7f5b181540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.914+0000 7f5b1de00640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1813bfe0 0x7f5b181425a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5b10008fd0 tx=0x7f5b100015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:39.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.914+0000 7f5b15600640 1 -- 172.21.15.49:0/3549765203 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10001b30 con 0x7f5b1813bfe0 2024-04-15T15:20:39.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.914+0000 7f5b15600640 1 -- 172.21.15.49:0/3549765203 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b10043070 con 0x7f5b1813bfe0 2024-04-15T15:20:39.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.914+0000 7f5b1e800640 1 -- 172.21.15.49:0/3549765203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b181606f0 con 0x7f5b1813bfe0 2024-04-15T15:20:39.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.914+0000 7f5b15600640 1 -- 172.21.15.49:0/3549765203 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10003b90 con 0x7f5b1813bfe0 2024-04-15T15:20:39.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.914+0000 7f5b1e800640 1 -- 172.21.15.49:0/3549765203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1813b500 con 0x7f5b1813bfe0 2024-04-15T15:20:39.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.915+0000 7f5b1e800640 1 -- 172.21.15.49:0/3549765203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b18072820 con 0x7f5b1813bfe0 2024-04-15T15:20:39.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.916+0000 7f5b15600640 1 -- 172.21.15.49:0/3549765203 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5b10005070 con 0x7f5b1813bfe0 2024-04-15T15:20:39.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.917+0000 7f5b15600640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5af806c9b0 0x7f5af806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:39.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.918+0000 7f5b15600640 1 -- 172.21.15.49:0/3549765203 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f5b100adca0 con 0x7f5b1813bfe0 2024-04-15T15:20:39.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.918+0000 7f5b1d400640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5af806c9b0 0x7f5af806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:39.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.919+0000 7f5b1d400640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5af806c9b0 0x7f5af806ee70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c001c10 tx=0x7f5b0c000f30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:39.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:39.922+0000 7f5b15600640 1 -- 172.21.15.49:0/3549765203 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b1007bde0 con 0x7f5b1813bfe0 2024-04-15T15:20:40.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.139+0000 7f5b1e800640 1 -- 172.21.15.49:0/3549765203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5b18072260 con 0x7f5b1813bfe0 2024-04-15T15:20:40.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.140+0000 7f5b15600640 1 -- 172.21.15.49:0/3549765203 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) ==== 80+0+122 (secure 0 0 0) 0x7f5b1007b6e0 con 0x7f5b1813bfe0 2024-04-15T15:20:40.142 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6837/1023024717 2024-04-16T15:20:28.618766+0000 2024-04-15T15:20:40.143 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6836/1023024717 2024-04-16T15:20:28.618766+0000 2024-04-15T15:20:40.143 INFO:teuthology.orchestra.run.smithi049.stderr:listed 2 entries 2024-04-15T15:20:40.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.143+0000 7f5b1e800640 1 -- 172.21.15.49:0/3549765203 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5af806c9b0 msgr2=0x7f5af806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.143+0000 7f5b1e800640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5af806c9b0 0x7f5af806ee70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c001c10 tx=0x7f5b0c000f30 comp rx=0 tx=0).stop 2024-04-15T15:20:40.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.143+0000 7f5b1e800640 1 -- 172.21.15.49:0/3549765203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1813bfe0 msgr2=0x7f5b181425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.143+0000 7f5b1e800640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1813bfe0 0x7f5b181425a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5b10008fd0 tx=0x7f5b100015b0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.143+0000 7f5b1e800640 1 -- 172.21.15.49:0/3549765203 shutdown_connections 2024-04-15T15:20:40.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.143+0000 7f5b1e800640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5af806c9b0 0x7f5af806ee70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.143+0000 7f5b1e800640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1813bfe0 0x7f5b181425a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.143+0000 7f5b1e800640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b1815e2c0 0x7f5b18161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.143+0000 7f5b1e800640 1 --2- 172.21.15.49:0/3549765203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b00004d40 0x7f5b181540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.143+0000 7f5b1e800640 1 -- 172.21.15.49:0/3549765203 >> 172.21.15.49:0/3549765203 conn(0x7f5b180b23a0 msgr2=0x7f5b18145020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:40.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.144+0000 7f5b1e800640 1 -- 172.21.15.49:0/3549765203 shutdown_connections 2024-04-15T15:20:40.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.144+0000 7f5b1e800640 1 -- 172.21.15.49:0/3549765203 wait complete. 2024-04-15T15:20:40.165 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-15T15:20:40.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:40.165 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-15T15:20:40.280 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:20:40.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:40.280 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-15T15:20:40.333 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:20:40.333 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:20:40.333 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:40.397 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:20:40.397 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:40.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:40.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.681+0000 7fd4dee00640 1 Processor -- start 2024-04-15T15:20:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.681+0000 7fd4dee00640 1 -- start start 2024-04-15T15:20:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.682+0000 7fd4dee00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd4d80cee00 0x7fd4d806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.682+0000 7fd4dee00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4d815d2f0 0x7fd4d80723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.682+0000 7fd4dee00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d815dbb0 con 0x7fd4d8071f50 2024-04-15T15:20:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.683+0000 7fd4dee00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d806f4f0 con 0x7fd4d815d2f0 2024-04-15T15:20:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.683+0000 7fd4dee00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d80728f0 con 0x7fd4d80cee00 2024-04-15T15:20:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.683+0000 7fd4de400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4d815d2f0 0x7fd4d80723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.683+0000 7fd4de400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4d815d2f0 0x7fd4d80723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41602/0 (socket says 172.21.15.49:41602) 2024-04-15T15:20:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.683+0000 7fd4de400640 1 -- 172.21.15.49:0/1872626921 learned_addr learned my addr 172.21.15.49:0/1872626921 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.683+0000 7fd4de400640 1 -- 172.21.15.49:0/1872626921 >> v2:172.21.15.107:3300/0 conn(0x7fd4d80cee00 msgr2=0x7fd4d806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.683+0000 7fd4de400640 1 --2- 172.21.15.49:0/1872626921 >> v2:172.21.15.107:3300/0 conn(0x7fd4d80cee00 0x7fd4d806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.683+0000 7fd4de400640 1 -- 172.21.15.49:0/1872626921 >> v1:172.21.15.107:6789/0 conn(0x7fd4d8071f50 legacy=0x7fd4d815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.683+0000 7fd4de400640 1 -- 172.21.15.49:0/1872626921 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d815d8d0 con 0x7fd4d815d2f0 2024-04-15T15:20:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.685+0000 7fd4de400640 1 --2- 172.21.15.49:0/1872626921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 0x7fd4d80723b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8070360 tx=0x7fd4d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755d0050840f99f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.685+0000 7fd4d7e00640 1 -- 172.21.15.49:0/1872626921 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d002ca80 con 0x7fd4d815d2f0 2024-04-15T15:20:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.685+0000 7fd4d7e00640 1 -- 172.21.15.49:0/1872626921 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d002cc20 con 0x7fd4d815d2f0 2024-04-15T15:20:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.685+0000 7fd4d7e00640 1 -- 172.21.15.49:0/1872626921 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00343f0 con 0x7fd4d815d2f0 2024-04-15T15:20:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.686+0000 7fd4dee00640 1 -- 172.21.15.49:0/1872626921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 msgr2=0x7fd4d80723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.686+0000 7fd4dee00640 1 --2- 172.21.15.49:0/1872626921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 0x7fd4d80723b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8070360 tx=0x7fd4d002bee0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.686+0000 7fd4dee00640 1 -- 172.21.15.49:0/1872626921 shutdown_connections 2024-04-15T15:20:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.686+0000 7fd4dee00640 1 --2- 172.21.15.49:0/1872626921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 0x7fd4d80723b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.686+0000 7fd4dee00640 1 --2- 172.21.15.49:0/1872626921 >> v2:172.21.15.107:3300/0 conn(0x7fd4d80cee00 0x7fd4d806e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.686+0000 7fd4dee00640 1 -- 172.21.15.49:0/1872626921 >> 172.21.15.49:0/1872626921 conn(0x7fd4d806b930 msgr2=0x7fd4d806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.686+0000 7fd4dee00640 1 -- 172.21.15.49:0/1872626921 shutdown_connections 2024-04-15T15:20:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.686+0000 7fd4dee00640 1 -- 172.21.15.49:0/1872626921 wait complete. 2024-04-15T15:20:40.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.688+0000 7fd4dee00640 1 Processor -- start 2024-04-15T15:20:40.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.688+0000 7fd4dee00640 1 -- start start 2024-04-15T15:20:40.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.689+0000 7fd4dee00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d8071f50 0x7fd4d8152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.689+0000 7fd4dee00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d80cee00 0x7fd4d81607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.689+0000 7fd4dee00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 0x7fd4d81f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.689+0000 7fd4dee00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d806e3a0 con 0x7fd4d815d2f0 2024-04-15T15:20:40.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.689+0000 7fd4dee00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d806e0c0 con 0x7fd4d8071f50 2024-04-15T15:20:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.689+0000 7fd4dee00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d815dda0 con 0x7fd4d80cee00 2024-04-15T15:20:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.689+0000 7fd4de400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 0x7fd4d81f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.689+0000 7fd4de400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 0x7fd4d81f4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41612/0 (socket says 172.21.15.49:41612) 2024-04-15T15:20:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.689+0000 7fd4de400640 1 -- 172.21.15.49:0/3134609097 learned_addr learned my addr 172.21.15.49:0/3134609097 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.690+0000 7fd4dda00640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d8071f50 0x7fd4d8152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.690+0000 7fd4de400640 1 -- 172.21.15.49:0/3134609097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d80cee00 msgr2=0x7fd4d81607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.690+0000 7fd4de400640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d80cee00 0x7fd4d81607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.690+0000 7fd4de400640 1 -- 172.21.15.49:0/3134609097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d8071f50 msgr2=0x7fd4d8152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.690+0000 7fd4de400640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d8071f50 0x7fd4d8152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.690+0000 7fd4de400640 1 -- 172.21.15.49:0/3134609097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d81f5560 con 0x7fd4d815d2f0 2024-04-15T15:20:40.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.690+0000 7fd4de400640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 0x7fd4d81f4de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0036040 tx=0x7fd4d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:40.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.690+0000 7fd4d5600640 1 -- 172.21.15.49:0/3134609097 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0043020 con 0x7fd4d815d2f0 2024-04-15T15:20:40.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.690+0000 7fd4d5600640 1 -- 172.21.15.49:0/3134609097 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d003e450 con 0x7fd4d815d2f0 2024-04-15T15:20:40.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.691+0000 7fd4dee00640 1 -- 172.21.15.49:0/3134609097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d81f5840 con 0x7fd4d815d2f0 2024-04-15T15:20:40.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.691+0000 7fd4d5600640 1 -- 172.21.15.49:0/3134609097 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d003e840 con 0x7fd4d815d2f0 2024-04-15T15:20:40.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.691+0000 7fd4dee00640 1 -- 172.21.15.49:0/3134609097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d81f5e00 con 0x7fd4d815d2f0 2024-04-15T15:20:40.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.692+0000 7fd4dee00640 1 -- 172.21.15.49:0/3134609097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d80755c0 con 0x7fd4d815d2f0 2024-04-15T15:20:40.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.693+0000 7fd4d5600640 1 -- 172.21.15.49:0/3134609097 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd4d0046020 con 0x7fd4d815d2f0 2024-04-15T15:20:40.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.694+0000 7fd4d5600640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd4b806c980 0x7fd4b806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:40.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.694+0000 7fd4d5600640 1 -- 172.21.15.49:0/3134609097 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fd4d00ae670 con 0x7fd4d815d2f0 2024-04-15T15:20:40.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.694+0000 7fd4dda00640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd4b806c980 0x7fd4b806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:40.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.695+0000 7fd4dda00640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd4b806c980 0x7fd4b806ee40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009890 tx=0x7fd4cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:40.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.699+0000 7fd4d5600640 1 -- 172.21.15.49:0/3134609097 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4d004d030 con 0x7fd4d815d2f0 2024-04-15T15:20:40.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.907+0000 7fd4dee00640 1 -- 172.21.15.49:0/3134609097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4d80afb60 con 0x7fd4d815d2f0 2024-04-15T15:20:40.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.909+0000 7fd4d5600640 1 -- 172.21.15.49:0/3134609097 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16798 (secure 0 0 0) 0x7fd4d007c950 con 0x7fd4d815d2f0 2024-04-15T15:20:40.912 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:40.912 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:28.618835+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6837/1023024717":"2024-04-16T15:20:28.618766+0000","172.21.15.49:6836/1023024717":"2024-04-16T15:20:28.618766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 -- 172.21.15.49:0/3134609097 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd4b806c980 msgr2=0x7fd4b806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd4b806c980 0x7fd4b806ee40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009890 tx=0x7fd4cc008040 comp rx=0 tx=0).stop 2024-04-15T15:20:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 -- 172.21.15.49:0/3134609097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 msgr2=0x7fd4d81f4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 0x7fd4d81f4de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0036040 tx=0x7fd4d0004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 -- 172.21.15.49:0/3134609097 shutdown_connections 2024-04-15T15:20:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd4b806c980 0x7fd4b806ee40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d80cee00 0x7fd4d81607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d815d2f0 0x7fd4d81f4de0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 --2- 172.21.15.49:0/3134609097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d8071f50 0x7fd4d8152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 -- 172.21.15.49:0/3134609097 >> 172.21.15.49:0/3134609097 conn(0x7fd4d806b930 msgr2=0x7fd4d80cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 -- 172.21.15.49:0/3134609097 shutdown_connections 2024-04-15T15:20:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:40.913+0000 7fd4dee00640 1 -- 172.21.15.49:0/3134609097 wait complete. 2024-04-15T15:20:40.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:58.063498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:59.009342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:40.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:41.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.165+0000 7f6cb3200640 1 Processor -- start 2024-04-15T15:20:41.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.165+0000 7f6cb3200640 1 -- start start 2024-04-15T15:20:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.167+0000 7f6cb3200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6cac15d510 0x7f6cac06ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.167+0000 7f6cb3200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6cac0b59b0 0x7f6cac0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.167+0000 7f6cb3200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cac0b6300 con 0x7f6cac15d8f0 2024-04-15T15:20:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.167+0000 7f6cb3200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f6cac0cea20 con 0x7f6cac15d510 2024-04-15T15:20:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.167+0000 7f6cb3200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f6cac0cecd0 con 0x7f6cac0b59b0 2024-04-15T15:20:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.167+0000 7f6cb1e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6cac15d510 0x7f6cac06ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.167+0000 7f6cb1e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6cac15d510 0x7f6cac06ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41620/0 (socket says 172.21.15.49:41620) 2024-04-15T15:20:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.167+0000 7f6cb1e00640 1 -- 172.21.15.49:0/1875174440 learned_addr learned my addr 172.21.15.49:0/1875174440 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.167+0000 7f6cb1400640 1 --1- 172.21.15.49:0/1875174440 >> v1:172.21.15.49:6789/0 conn(0x7f6cac15d8f0 0x7f6cac0ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60126/0 (socket says 172.21.15.49:60126) 2024-04-15T15:20:41.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.168+0000 7f6cb2800640 1 --2- 172.21.15.49:0/1875174440 >> v2:172.21.15.107:3300/0 conn(0x7f6cac0b59b0 0x7f6cac0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.168+0000 7f6cb1e00640 1 -- 172.21.15.49:0/1875174440 >> v2:172.21.15.107:3300/0 conn(0x7f6cac0b59b0 msgr2=0x7f6cac0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:41.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.168+0000 7f6cb1e00640 1 --2- 172.21.15.49:0/1875174440 >> v2:172.21.15.107:3300/0 conn(0x7f6cac0b59b0 0x7f6cac0b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.168+0000 7f6cb1e00640 1 -- 172.21.15.49:0/1875174440 >> v1:172.21.15.49:6789/0 conn(0x7f6cac15d8f0 legacy=0x7f6cac0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.168+0000 7f6cb0a00640 1 -- 172.21.15.49:0/1875174440 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3277830665 0 0) 0x7f6c9c001110 con 0x7f6cac15d8f0 2024-04-15T15:20:41.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.168+0000 7f6cb1e00640 1 -- 172.21.15.49:0/1875174440 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac0cefb0 con 0x7f6cac15d510 2024-04-15T15:20:41.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.168+0000 7f6cb2800640 1 --2- 172.21.15.49:0/1875174440 >> v2:172.21.15.107:3300/0 conn(0x7f6cac0b59b0 0x7f6cac0b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.169+0000 7f6cb1e00640 1 --2- 172.21.15.49:0/1875174440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 0x7f6cac06ea80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0703b0 tx=0x7f6ca002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84c3cf4c93ecea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.169+0000 7f6cb0a00640 1 -- 172.21.15.49:0/1875174440 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0034070 con 0x7f6cac15d510 2024-04-15T15:20:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.169+0000 7f6cb0a00640 1 -- 172.21.15.49:0/1875174440 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca0030860 con 0x7f6cac15d510 2024-04-15T15:20:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.169+0000 7f6cb0a00640 1 -- 172.21.15.49:0/1875174440 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0030c30 con 0x7f6cac15d510 2024-04-15T15:20:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.170+0000 7f6cb3200640 1 -- 172.21.15.49:0/1875174440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 msgr2=0x7f6cac06ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.170+0000 7f6cb3200640 1 --2- 172.21.15.49:0/1875174440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 0x7f6cac06ea80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0703b0 tx=0x7f6ca002b9a0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.170+0000 7f6cb3200640 1 -- 172.21.15.49:0/1875174440 shutdown_connections 2024-04-15T15:20:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.170+0000 7f6cb3200640 1 --2- 172.21.15.49:0/1875174440 >> v2:172.21.15.107:3300/0 conn(0x7f6cac0b59b0 0x7f6cac0b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.170+0000 7f6cb3200640 1 --2- 172.21.15.49:0/1875174440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 0x7f6cac06ea80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.170+0000 7f6cb3200640 1 -- 172.21.15.49:0/1875174440 >> 172.21.15.49:0/1875174440 conn(0x7f6cac0b23a0 msgr2=0x7f6cac0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.170+0000 7f6cb3200640 1 -- 172.21.15.49:0/1875174440 shutdown_connections 2024-04-15T15:20:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.170+0000 7f6cb3200640 1 -- 172.21.15.49:0/1875174440 wait complete. 2024-04-15T15:20:41.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.172+0000 7f6cb3200640 1 Processor -- start 2024-04-15T15:20:41.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.172+0000 7f6cb3200640 1 -- start start 2024-04-15T15:20:41.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.173+0000 7f6cb3200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cac0b59b0 0x7f6cac14fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.173+0000 7f6cb3200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 0x7f6cac0b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.173+0000 7f6cb3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac0aeea0 0x7f6cac0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.173+0000 7f6cb3200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac06e560 con 0x7f6cac15d510 2024-04-15T15:20:41.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.173+0000 7f6cb3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac06e280 con 0x7f6cac0aeea0 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.173+0000 7f6cb3200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6cac06efc0 con 0x7f6cac0b59b0 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.173+0000 7f6cb1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 0x7f6cac0b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.174+0000 7f6cb1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 0x7f6cac0b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41630/0 (socket says 172.21.15.49:41630) 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.174+0000 7f6cb1400640 1 -- 172.21.15.49:0/3380163254 learned_addr learned my addr 172.21.15.49:0/3380163254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.174+0000 7f6cb2800640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac0aeea0 0x7f6cac0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.174+0000 7f6cb1e00640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cac0b59b0 0x7f6cac14fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.174+0000 7f6cb1400640 1 -- 172.21.15.49:0/3380163254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cac0b59b0 msgr2=0x7f6cac14fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.174+0000 7f6cb1400640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cac0b59b0 0x7f6cac14fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.174+0000 7f6cb1400640 1 -- 172.21.15.49:0/3380163254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac0aeea0 msgr2=0x7f6cac0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.174+0000 7f6cb1400640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac0aeea0 0x7f6cac0b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.174+0000 7f6cb1400640 1 -- 172.21.15.49:0/3380163254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac150270 con 0x7f6cac15d510 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.174+0000 7f6cb1400640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 0x7f6cac0b4950 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00c780 tx=0x7f6c9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:41.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.175+0000 7f6ca6000640 1 -- 172.21.15.49:0/3380163254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c018070 con 0x7f6cac15d510 2024-04-15T15:20:41.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.175+0000 7f6ca6000640 1 -- 172.21.15.49:0/3380163254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c9c0072d0 con 0x7f6cac15d510 2024-04-15T15:20:41.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.175+0000 7f6cb3200640 1 -- 172.21.15.49:0/3380163254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cac1e7b60 con 0x7f6cac15d510 2024-04-15T15:20:41.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.175+0000 7f6ca6000640 1 -- 172.21.15.49:0/3380163254 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c0047e0 con 0x7f6cac15d510 2024-04-15T15:20:41.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.175+0000 7f6cb3200640 1 -- 172.21.15.49:0/3380163254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cac1e8120 con 0x7f6cac15d510 2024-04-15T15:20:41.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.176+0000 7f6cb3200640 1 -- 172.21.15.49:0/3380163254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cac0755c0 con 0x7f6cac15d510 2024-04-15T15:20:41.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.177+0000 7f6ca6000640 1 -- 172.21.15.49:0/3380163254 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6c9c01b070 con 0x7f6cac15d510 2024-04-15T15:20:41.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.179+0000 7f6ca6000640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c9006c9b0 0x7f6c9006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.179+0000 7f6ca6000640 1 -- 172.21.15.49:0/3380163254 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f6c9c08afd0 con 0x7f6cac15d510 2024-04-15T15:20:41.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.180+0000 7f6cb1e00640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c9006c9b0 0x7f6c9006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.180+0000 7f6cb1e00640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c9006c9b0 0x7f6c9006ee70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0036000 tx=0x7f6ca003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:41.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.184+0000 7f6ca6000640 1 -- 172.21.15.49:0/3380163254 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cac0b6300 con 0x7f6cac15d510 2024-04-15T15:20:41.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.386+0000 7f6cb3200640 1 -- 172.21.15.49:0/3380163254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6cac074f80 con 0x7f6cac15d510 2024-04-15T15:20:41.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.387+0000 7f6ca6000640 1 -- 172.21.15.49:0/3380163254 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f6c9c059340 con 0x7f6cac15d510 2024-04-15T15:20:41.389 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:20:41.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 -- 172.21.15.49:0/3380163254 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c9006c9b0 msgr2=0x7f6c9006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c9006c9b0 0x7f6c9006ee70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0036000 tx=0x7f6ca003b040 comp rx=0 tx=0).stop 2024-04-15T15:20:41.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 -- 172.21.15.49:0/3380163254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 msgr2=0x7f6cac0b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 0x7f6cac0b4950 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00c780 tx=0x7f6c9c00cc40 comp rx=0 tx=0).stop 2024-04-15T15:20:41.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 -- 172.21.15.49:0/3380163254 shutdown_connections 2024-04-15T15:20:41.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6c9006c9b0 0x7f6c9006ee70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac0aeea0 0x7f6cac0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac15d510 0x7f6cac0b4950 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 --2- 172.21.15.49:0/3380163254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cac0b59b0 0x7f6cac14fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 -- 172.21.15.49:0/3380163254 >> 172.21.15.49:0/3380163254 conn(0x7f6cac0b23a0 msgr2=0x7f6cac15b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:41.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 -- 172.21.15.49:0/3380163254 shutdown_connections 2024-04-15T15:20:41.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.390+0000 7f6cb3200640 1 -- 172.21.15.49:0/3380163254 wait complete. 2024-04-15T15:20:41.414 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:41.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.650+0000 7f175e400640 1 Processor -- start 2024-04-15T15:20:41.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.650+0000 7f175e400640 1 -- start start 2024-04-15T15:20:41.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.652+0000 7f175e400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f17581461c0 0x7f1758071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.652+0000 7f175e400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f17580cebb0 0x7f175806df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.652+0000 7f175e400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17580cd2d0 con 0x7f1758072420 2024-04-15T15:20:41.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.652+0000 7f175e400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f17580cf1d0 con 0x7f17580cebb0 2024-04-15T15:20:41.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.652+0000 7f175e400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f1758072880 con 0x7f17581461c0 2024-04-15T15:20:41.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.652+0000 7f1757e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1758072420 0x7f17580ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60140/0 (socket says 172.21.15.49:60140) 2024-04-15T15:20:41.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.652+0000 7f1757e00640 1 -- 172.21.15.49:0/1875299341 learned_addr learned my addr 172.21.15.49:0/1875299341 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:41.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.652+0000 7f175da00640 1 --2- 172.21.15.49:0/1875299341 >> v2:172.21.15.49:3300/0 conn(0x7f17580cebb0 0x7f175806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.652+0000 7f175d000640 1 --2- 172.21.15.49:0/1875299341 >> v2:172.21.15.107:3300/0 conn(0x7f17581461c0 0x7f1758071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.653+0000 7f175da00640 1 -- 172.21.15.49:0/1875299341 >> v2:172.21.15.107:3300/0 conn(0x7f17581461c0 msgr2=0x7f1758071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:41.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.653+0000 7f175da00640 1 --2- 172.21.15.49:0/1875299341 >> v2:172.21.15.107:3300/0 conn(0x7f17581461c0 0x7f1758071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.653+0000 7f175da00640 1 -- 172.21.15.49:0/1875299341 >> v1:172.21.15.49:6789/0 conn(0x7f1758072420 legacy=0x7f17580ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.653+0000 7f175da00640 1 -- 172.21.15.49:0/1875299341 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17580cd5b0 con 0x7f17580cebb0 2024-04-15T15:20:41.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.654+0000 7f175da00640 1 --2- 172.21.15.49:0/1875299341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17580cebb0 0x7f175806df20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f17580d05c0 tx=0x7f175002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6d84e720ad507f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:41.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.655+0000 7f1757400640 1 -- 172.21.15.49:0/1875299341 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750039070 con 0x7f17580cebb0 2024-04-15T15:20:41.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.655+0000 7f1757400640 1 -- 172.21.15.49:0/1875299341 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175002cc70 con 0x7f17580cebb0 2024-04-15T15:20:41.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.655+0000 7f1757400640 1 -- 172.21.15.49:0/1875299341 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750034660 con 0x7f17580cebb0 2024-04-15T15:20:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.655+0000 7f175e400640 1 -- 172.21.15.49:0/1875299341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17580cebb0 msgr2=0x7f175806df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.655+0000 7f175e400640 1 --2- 172.21.15.49:0/1875299341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17580cebb0 0x7f175806df20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f17580d05c0 tx=0x7f175002bec0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.655+0000 7f175e400640 1 -- 172.21.15.49:0/1875299341 shutdown_connections 2024-04-15T15:20:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.655+0000 7f175e400640 1 --2- 172.21.15.49:0/1875299341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17580cebb0 0x7f175806df20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.655+0000 7f175e400640 1 --2- 172.21.15.49:0/1875299341 >> v2:172.21.15.107:3300/0 conn(0x7f17581461c0 0x7f1758071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.655+0000 7f175e400640 1 -- 172.21.15.49:0/1875299341 >> 172.21.15.49:0/1875299341 conn(0x7f175806b930 msgr2=0x7f175806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.655+0000 7f175e400640 1 -- 172.21.15.49:0/1875299341 shutdown_connections 2024-04-15T15:20:41.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.656+0000 7f175e400640 1 -- 172.21.15.49:0/1875299341 wait complete. 2024-04-15T15:20:41.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.657+0000 7f175e400640 1 Processor -- start 2024-04-15T15:20:41.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.658+0000 7f175e400640 1 -- start start 2024-04-15T15:20:41.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.658+0000 7f175e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1758072420 0x7f17581f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175e400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17580cebb0 0x7f17580b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175e400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17581461c0 0x7f17580b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175e400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17580cdb70 con 0x7f17581461c0 2024-04-15T15:20:41.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175e400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17580cd080 con 0x7f1758072420 2024-04-15T15:20:41.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175e400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f175806e460 con 0x7f17580cebb0 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175da00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17581461c0 0x7f17580b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175da00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17581461c0 0x7f17580b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41656/0 (socket says 172.21.15.49:41656) 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175da00640 1 -- 172.21.15.49:0/3494234589 learned_addr learned my addr 172.21.15.49:0/3494234589 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175d000640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1758072420 0x7f17581f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f1757e00640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17580cebb0 0x7f17580b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175da00640 1 -- 172.21.15.49:0/3494234589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17580cebb0 msgr2=0x7f17580b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175da00640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17580cebb0 0x7f17580b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175da00640 1 -- 172.21.15.49:0/3494234589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1758072420 msgr2=0x7f17581f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175da00640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1758072420 0x7f17581f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f175da00640 1 -- 172.21.15.49:0/3494234589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17581e7230 con 0x7f17581461c0 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.659+0000 7f1757e00640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17580cebb0 0x7f17580b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.660+0000 7f175da00640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17581461c0 0x7f17580b4c40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1750009e80 tx=0x7f1750004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:41.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.660+0000 7f1754c00640 1 -- 172.21.15.49:0/3494234589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750039050 con 0x7f17581461c0 2024-04-15T15:20:41.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.660+0000 7f1754c00640 1 -- 172.21.15.49:0/3494234589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1750045070 con 0x7f17581461c0 2024-04-15T15:20:41.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.660+0000 7f175e400640 1 -- 172.21.15.49:0/3494234589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17581e7510 con 0x7f17581461c0 2024-04-15T15:20:41.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.660+0000 7f1754c00640 1 -- 172.21.15.49:0/3494234589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750009450 con 0x7f17581461c0 2024-04-15T15:20:41.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.660+0000 7f175e400640 1 -- 172.21.15.49:0/3494234589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17581e7ad0 con 0x7f17581461c0 2024-04-15T15:20:41.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.661+0000 7f175e400640 1 -- 172.21.15.49:0/3494234589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17580755c0 con 0x7f17581461c0 2024-04-15T15:20:41.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.663+0000 7f1754c00640 1 -- 172.21.15.49:0/3494234589 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f175003e850 con 0x7f17581461c0 2024-04-15T15:20:41.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.664+0000 7f1754c00640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f173c06c980 0x7f173c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:41.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.664+0000 7f1754c00640 1 -- 172.21.15.49:0/3494234589 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f17500ae940 con 0x7f17581461c0 2024-04-15T15:20:41.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.664+0000 7f175d000640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f173c06c980 0x7f173c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:41.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.665+0000 7f175d000640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f173c06c980 0x7f173c06ee40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1758144e10 tx=0x7f174c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:41.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.668+0000 7f1754c00640 1 -- 172.21.15.49:0/3494234589 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f175002ce20 con 0x7f17581461c0 2024-04-15T15:20:41.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.874+0000 7f175e400640 1 -- 172.21.15.49:0/3494234589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1758072e40 con 0x7f17581461c0 2024-04-15T15:20:41.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.875+0000 7f1754c00640 1 -- 172.21.15.49:0/3494234589 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7f175007cc20 con 0x7f17581461c0 2024-04-15T15:20:41.877 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:20:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 -- 172.21.15.49:0/3494234589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f173c06c980 msgr2=0x7f173c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f173c06c980 0x7f173c06ee40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1758144e10 tx=0x7f174c008040 comp rx=0 tx=0).stop 2024-04-15T15:20:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 -- 172.21.15.49:0/3494234589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17581461c0 msgr2=0x7f17580b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17581461c0 0x7f17580b4c40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1750009e80 tx=0x7f1750004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 -- 172.21.15.49:0/3494234589 shutdown_connections 2024-04-15T15:20:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f173c06c980 0x7f173c06ee40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17580cebb0 0x7f17580b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17581461c0 0x7f17580b4c40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 --2- 172.21.15.49:0/3494234589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1758072420 0x7f17581f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 -- 172.21.15.49:0/3494234589 >> 172.21.15.49:0/3494234589 conn(0x7f175806b930 msgr2=0x7f1758156ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 -- 172.21.15.49:0/3494234589 shutdown_connections 2024-04-15T15:20:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:41.878+0000 7f175e400640 1 -- 172.21.15.49:0/3494234589 wait complete. 2024-04-15T15:20:41.899 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.130+0000 7f8f82c00640 1 Processor -- start 2024-04-15T15:20:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.130+0000 7f8f82c00640 1 -- start start 2024-04-15T15:20:42.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.131+0000 7f8f82c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8f7c15c680 0x7f8f7c162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.131+0000 7f8f82c00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7c0b70d0 con 0x7f8f7c0b6c70 2024-04-15T15:20:42.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.131+0000 7f8f82c00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7c162dd0 con 0x7f8f7c0d2b50 2024-04-15T15:20:42.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.131+0000 7f8f82c00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f8f7c1630b0 con 0x7f8f7c15c680 2024-04-15T15:20:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.132+0000 7f8f80e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8f7c0b6c70 0x7f8f7c15f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60146/0 (socket says 172.21.15.49:60146) 2024-04-15T15:20:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.132+0000 7f8f80e00640 1 -- 172.21.15.49:0/2231339113 learned_addr learned my addr 172.21.15.49:0/2231339113 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.132+0000 7f8f81800640 1 --2- 172.21.15.49:0/2231339113 >> v2:172.21.15.107:3300/0 conn(0x7f8f7c15c680 0x7f8f7c162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.132+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2634703737 0 0) 0x7f8f7c0b70d0 con 0x7f8f7c0b6c70 2024-04-15T15:20:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.132+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f64003520 con 0x7f8f7c0b6c70 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2174673838 0 0) 0x7f8f64003520 con 0x7f8f7c0b6c70 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 >> v2:172.21.15.107:3300/0 conn(0x7f8f7c15c680 msgr2=0x7f8f7c162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 --2- 172.21.15.49:0/2231339113 >> v2:172.21.15.107:3300/0 conn(0x7f8f7c15c680 0x7f8f7c162730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 >> v1:172.21.15.107:6789/0 conn(0x7f8f7c0d2b50 legacy=0x7f8f7c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c163390 con 0x7f8f7c0b6c70 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f8f68001670 con 0x7f8f7c0b6c70 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 >> v1:172.21.15.49:6789/0 conn(0x7f8f7c0b6c70 legacy=0x7f8f7c15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f64003a00 0x7f8f64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f640051c0 0x7f8f6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1630b0 con 0x7f8f64004800 2024-04-15T15:20:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6400c0c0 con 0x7f8f640051c0 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6400c2c0 con 0x7f8f64003a00 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.133+0000 7f8f80e00640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.134+0000 7f8f82200640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f640051c0 0x7f8f6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.134+0000 7f8f81800640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f64003a00 0x7f8f64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.134+0000 7f8f80e00640 1 -- 172.21.15.49:0/2231339113 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f64003a00 msgr2=0x7f8f64003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.134+0000 7f8f80e00640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f64003a00 0x7f8f64003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.134+0000 7f8f80e00640 1 -- 172.21.15.49:0/2231339113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f640051c0 msgr2=0x7f8f6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.134+0000 7f8f80e00640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f640051c0 0x7f8f6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.134+0000 7f8f80e00640 1 -- 172.21.15.49:0/2231339113 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f64003520 con 0x7f8f64004800 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.134+0000 7f8f82200640 1 -- 172.21.15.49:0/2231339113 reap_dead start 2024-04-15T15:20:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.134+0000 7f8f81800640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f64003a00 0x7f8f64003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.135+0000 7f8f80e00640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f64004c80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800c110 tx=0x7f8f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf332048ee4f816a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.135+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68007c40 con 0x7f8f64004800 2024-04-15T15:20:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.135+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f68030070 con 0x7f8f64004800 2024-04-15T15:20:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.135+0000 7f8f7be00640 1 -- 172.21.15.49:0/2231339113 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68005d70 con 0x7f8f64004800 2024-04-15T15:20:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.136+0000 7f8f82c00640 1 -- 172.21.15.49:0/2231339113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 msgr2=0x7f8f64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.136+0000 7f8f82c00640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f64004c80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800c110 tx=0x7f8f64005830 comp rx=0 tx=0).stop 2024-04-15T15:20:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.136+0000 7f8f82c00640 1 -- 172.21.15.49:0/2231339113 shutdown_connections 2024-04-15T15:20:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.136+0000 7f8f82c00640 1 --2- 172.21.15.49:0/2231339113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f64004c80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.136+0000 7f8f82c00640 1 -- 172.21.15.49:0/2231339113 >> 172.21.15.49:0/2231339113 conn(0x7f8f7c073480 msgr2=0x7f8f7c073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.136+0000 7f8f82c00640 1 -- 172.21.15.49:0/2231339113 shutdown_connections 2024-04-15T15:20:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.136+0000 7f8f82c00640 1 -- 172.21.15.49:0/2231339113 wait complete. 2024-04-15T15:20:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.138+0000 7f8f82c00640 1 Processor -- start 2024-04-15T15:20:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.138+0000 7f8f82c00640 1 -- start start 2024-04-15T15:20:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.139+0000 7f8f82c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f7c1f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.139+0000 7f8f82c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f7c15b1b0 0x7f8f7c0b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.139+0000 7f8f82c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f7c0b8df0 0x7f8f7c1e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.139+0000 7f8f82c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c162730 con 0x7f8f64004800 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.139+0000 7f8f82c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c15ca60 con 0x7f8f7c0b8df0 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.139+0000 7f8f82c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c1628b0 con 0x7f8f7c15b1b0 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.139+0000 7f8f81800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f7c1f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f81800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f7c1f7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41670/0 (socket says 172.21.15.49:41670) 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f81800640 1 -- 172.21.15.49:0/1870700326 learned_addr learned my addr 172.21.15.49:0/1870700326 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f82200640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f7c0b8df0 0x7f8f7c1e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f80e00640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f7c15b1b0 0x7f8f7c0b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f81800640 1 -- 172.21.15.49:0/1870700326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f7c15b1b0 msgr2=0x7f8f7c0b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f81800640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f7c15b1b0 0x7f8f7c0b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f81800640 1 -- 172.21.15.49:0/1870700326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f7c0b8df0 msgr2=0x7f8f7c1e8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f81800640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f7c0b8df0 0x7f8f7c1e8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f81800640 1 -- 172.21.15.49:0/1870700326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c1f7fe0 con 0x7f8f64004800 2024-04-15T15:20:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f82200640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f7c0b8df0 0x7f8f7c1e8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.140+0000 7f8f81800640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f7c1f7aa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8f700107f0 tx=0x7f8f70010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.141+0000 7f8f79600640 1 -- 172.21.15.49:0/1870700326 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000f040 con 0x7f8f64004800 2024-04-15T15:20:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.141+0000 7f8f79600640 1 -- 172.21.15.49:0/1870700326 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7000eb60 con 0x7f8f64004800 2024-04-15T15:20:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.141+0000 7f8f82c00640 1 -- 172.21.15.49:0/1870700326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c0b92b0 con 0x7f8f64004800 2024-04-15T15:20:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.141+0000 7f8f79600640 1 -- 172.21.15.49:0/1870700326 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70010ea0 con 0x7f8f64004800 2024-04-15T15:20:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.141+0000 7f8f82c00640 1 -- 172.21.15.49:0/1870700326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c0b9870 con 0x7f8f64004800 2024-04-15T15:20:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.142+0000 7f8f82c00640 1 -- 172.21.15.49:0/1870700326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c0b9f30 con 0x7f8f64004800 2024-04-15T15:20:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.143+0000 7f8f79600640 1 -- 172.21.15.49:0/1870700326 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8f7001c400 con 0x7f8f64004800 2024-04-15T15:20:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.144+0000 7f8f79600640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8f5c06c980 0x7f8f5c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.144+0000 7f8f79600640 1 -- 172.21.15.49:0/1870700326 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f8f7008c160 con 0x7f8f64004800 2024-04-15T15:20:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.144+0000 7f8f80e00640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8f5c06c980 0x7f8f5c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.145+0000 7f8f80e00640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8f5c06c980 0x7f8f5c06ee40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f68034000 tx=0x7f8f68039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.149+0000 7f8f79600640 1 -- 172.21.15.49:0/1870700326 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f7c0b9f30 con 0x7f8f64004800 2024-04-15T15:20:42.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.357+0000 7f8f82c00640 1 -- 172.21.15.49:0/1870700326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8f7c06dbe0 con 0x7f8f64004800 2024-04-15T15:20:42.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.358+0000 7f8f79600640 1 -- 172.21.15.49:0/1870700326 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f8f70018030 con 0x7f8f64004800 2024-04-15T15:20:42.360 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 -- 172.21.15.49:0/1870700326 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8f5c06c980 msgr2=0x7f8f5c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8f5c06c980 0x7f8f5c06ee40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f68034000 tx=0x7f8f68039040 comp rx=0 tx=0).stop 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 -- 172.21.15.49:0/1870700326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 msgr2=0x7f8f7c1f7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f7c1f7aa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8f700107f0 tx=0x7f8f70010cb0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 -- 172.21.15.49:0/1870700326 shutdown_connections 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8f5c06c980 0x7f8f5c06ee40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f7c15b1b0 0x7f8f7c0b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f7c0b8df0 0x7f8f7c1e8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 --2- 172.21.15.49:0/1870700326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f64004800 0x7f8f7c1f7aa0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 -- 172.21.15.49:0/1870700326 >> 172.21.15.49:0/1870700326 conn(0x7f8f7c073480 msgr2=0x7f8f7c0d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:42.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 -- 172.21.15.49:0/1870700326 shutdown_connections 2024-04-15T15:20:42.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.361+0000 7f8f82c00640 1 -- 172.21.15.49:0/1870700326 wait complete. 2024-04-15T15:20:42.385 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:42.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.616+0000 7efc99000640 1 Processor -- start 2024-04-15T15:20:42.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.616+0000 7efc99000640 1 -- start start 2024-04-15T15:20:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.617+0000 7efc99000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc940b0cb0 0x7efc940ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.617+0000 7efc99000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc9415d770 0x7efc940cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.617+0000 7efc99000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc940cd940 con 0x7efc940b59b0 2024-04-15T15:20:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.617+0000 7efc99000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7efc940ce550 con 0x7efc940b0cb0 2024-04-15T15:20:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.617+0000 7efc99000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7efc940cdc20 con 0x7efc9415d770 2024-04-15T15:20:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.618+0000 7efc93400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc940b0cb0 0x7efc940ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.618+0000 7efc93400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc940b0cb0 0x7efc940ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41674/0 (socket says 172.21.15.49:41674) 2024-04-15T15:20:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.618+0000 7efc93400640 1 -- 172.21.15.49:0/3044121664 learned_addr learned my addr 172.21.15.49:0/3044121664 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.618+0000 7efc8aa00640 1 --2- 172.21.15.49:0/3044121664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc9415d770 0x7efc940cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.618+0000 7efc93400640 1 -- 172.21.15.49:0/3044121664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc9415d770 msgr2=0x7efc940cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.618+0000 7efc93400640 1 --2- 172.21.15.49:0/3044121664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc9415d770 0x7efc940cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.618+0000 7efc93400640 1 -- 172.21.15.49:0/3044121664 >> v1:172.21.15.49:6789/0 conn(0x7efc940b59b0 legacy=0x7efc9415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.618+0000 7efc93400640 1 -- 172.21.15.49:0/3044121664 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc940cdf00 con 0x7efc940b0cb0 2024-04-15T15:20:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.620+0000 7efc93400640 1 --2- 172.21.15.49:0/3044121664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc940b0cb0 0x7efc940ce9e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efc84009870 tx=0x7efc8402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614ff9638dbdad53 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.620+0000 7efc92a00640 1 -- 172.21.15.49:0/3044121664 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84034070 con 0x7efc940b0cb0 2024-04-15T15:20:42.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.620+0000 7efc92a00640 1 -- 172.21.15.49:0/3044121664 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc84030840 con 0x7efc940b0cb0 2024-04-15T15:20:42.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.620+0000 7efc92a00640 1 -- 172.21.15.49:0/3044121664 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84030c10 con 0x7efc940b0cb0 2024-04-15T15:20:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.620+0000 7efc99000640 1 -- 172.21.15.49:0/3044121664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc940b0cb0 msgr2=0x7efc940ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.620+0000 7efc99000640 1 --2- 172.21.15.49:0/3044121664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc940b0cb0 0x7efc940ce9e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efc84009870 tx=0x7efc8402b980 comp rx=0 tx=0).stop 2024-04-15T15:20:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.621+0000 7efc99000640 1 -- 172.21.15.49:0/3044121664 shutdown_connections 2024-04-15T15:20:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.621+0000 7efc99000640 1 --2- 172.21.15.49:0/3044121664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc9415d770 0x7efc940cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.621+0000 7efc99000640 1 --2- 172.21.15.49:0/3044121664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc940b0cb0 0x7efc940ce9e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.621+0000 7efc99000640 1 -- 172.21.15.49:0/3044121664 >> 172.21.15.49:0/3044121664 conn(0x7efc940b23a0 msgr2=0x7efc940b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.621+0000 7efc99000640 1 -- 172.21.15.49:0/3044121664 shutdown_connections 2024-04-15T15:20:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.621+0000 7efc99000640 1 -- 172.21.15.49:0/3044121664 wait complete. 2024-04-15T15:20:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.623+0000 7efc99000640 1 Processor -- start 2024-04-15T15:20:42.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.623+0000 7efc99000640 1 -- start start 2024-04-15T15:20:42.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.624+0000 7efc99000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940b0cb0 0x7efc940adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.624+0000 7efc99000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc940b59b0 0x7efc94151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.624+0000 7efc99000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9415d770 0x7efc940afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.624+0000 7efc99000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efc940cef20 con 0x7efc9415d770 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.624+0000 7efc99000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc940b1090 con 0x7efc940b59b0 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.624+0000 7efc99000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7efc9414c860 con 0x7efc940b0cb0 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.624+0000 7efc93e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9415d770 0x7efc940afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.624+0000 7efc93e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9415d770 0x7efc940afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41688/0 (socket says 172.21.15.49:41688) 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.624+0000 7efc93e00640 1 -- 172.21.15.49:0/893228944 learned_addr learned my addr 172.21.15.49:0/893228944 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc93400640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940b0cb0 0x7efc940adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc8aa00640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc940b59b0 0x7efc94151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc93e00640 1 -- 172.21.15.49:0/893228944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940b0cb0 msgr2=0x7efc940adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc93e00640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940b0cb0 0x7efc940adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc93e00640 1 -- 172.21.15.49:0/893228944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc940b59b0 msgr2=0x7efc94151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc93e00640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc940b59b0 0x7efc94151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc93e00640 1 -- 172.21.15.49:0/893228944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc94151a60 con 0x7efc9415d770 2024-04-15T15:20:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc93e00640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9415d770 0x7efc940afb30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00c390 tx=0x7efc8c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc8be00640 1 -- 172.21.15.49:0/893228944 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c017070 con 0x7efc9415d770 2024-04-15T15:20:42.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc8be00640 1 -- 172.21.15.49:0/893228944 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc8c0052d0 con 0x7efc9415d770 2024-04-15T15:20:42.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc8be00640 1 -- 172.21.15.49:0/893228944 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c003810 con 0x7efc9415d770 2024-04-15T15:20:42.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.625+0000 7efc99000640 1 -- 172.21.15.49:0/893228944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc94157ee0 con 0x7efc9415d770 2024-04-15T15:20:42.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.626+0000 7efc99000640 1 -- 172.21.15.49:0/893228944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc94158720 con 0x7efc9415d770 2024-04-15T15:20:42.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.627+0000 7efc99000640 1 -- 172.21.15.49:0/893228944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc94158920 con 0x7efc9415d770 2024-04-15T15:20:42.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.628+0000 7efc8be00640 1 -- 172.21.15.49:0/893228944 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7efc8c01a070 con 0x7efc9415d770 2024-04-15T15:20:42.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.629+0000 7efc8be00640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc7006c980 0x7efc7006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:42.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.629+0000 7efc8be00640 1 -- 172.21.15.49:0/893228944 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7efc8c08afd0 con 0x7efc9415d770 2024-04-15T15:20:42.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.629+0000 7efc93400640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc7006c980 0x7efc7006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:42.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.630+0000 7efc93400640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc7006c980 0x7efc7006ee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc84036000 tx=0x7efc8403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:42.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.633+0000 7efc8be00640 1 -- 172.21.15.49:0/893228944 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc94158920 con 0x7efc9415d770 2024-04-15T15:20:42.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.853+0000 7efc99000640 1 -- 172.21.15.49:0/893228944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efc940755c0 con 0x7efc9415d770 2024-04-15T15:20:42.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.853+0000 7efc8be00640 1 -- 172.21.15.49:0/893228944 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) ==== 80+0+122 (secure 0 0 0) 0x7efc94157ee0 con 0x7efc9415d770 2024-04-15T15:20:42.855 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6837/1023024717 2024-04-16T15:20:28.618766+0000 2024-04-15T15:20:42.855 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6836/1023024717 2024-04-16T15:20:28.618766+0000 2024-04-15T15:20:42.856 INFO:teuthology.orchestra.run.smithi049.stderr:listed 2 entries 2024-04-15T15:20:42.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.856+0000 7efc99000640 1 -- 172.21.15.49:0/893228944 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc7006c980 msgr2=0x7efc7006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.856+0000 7efc99000640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc7006c980 0x7efc7006ee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc84036000 tx=0x7efc8403b040 comp rx=0 tx=0).stop 2024-04-15T15:20:42.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.856+0000 7efc99000640 1 -- 172.21.15.49:0/893228944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9415d770 msgr2=0x7efc940afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:42.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.856+0000 7efc99000640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9415d770 0x7efc940afb30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00c390 tx=0x7efc8c00c850 comp rx=0 tx=0).stop 2024-04-15T15:20:42.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.856+0000 7efc99000640 1 -- 172.21.15.49:0/893228944 shutdown_connections 2024-04-15T15:20:42.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.856+0000 7efc99000640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efc7006c980 0x7efc7006ee40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.856+0000 7efc99000640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9415d770 0x7efc940afb30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.856+0000 7efc99000640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc940b59b0 0x7efc94151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.856+0000 7efc99000640 1 --2- 172.21.15.49:0/893228944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940b0cb0 0x7efc940adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:42.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.856+0000 7efc99000640 1 -- 172.21.15.49:0/893228944 >> 172.21.15.49:0/893228944 conn(0x7efc940b23a0 msgr2=0x7efc9406f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:42.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.857+0000 7efc99000640 1 -- 172.21.15.49:0/893228944 shutdown_connections 2024-04-15T15:20:42.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:42.857+0000 7efc99000640 1 -- 172.21.15.49:0/893228944 wait complete. 2024-04-15T15:20:42.877 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-15T15:20:42.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:42.877 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.2 2024-04-15T15:20:42.993 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:20:42.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:42.994 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-15T15:20:43.049 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:20:43.049 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:20:43.049 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:43.107 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:20:43.107 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:43.161 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:43.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.398+0000 7f5f8d400640 1 Processor -- start 2024-04-15T15:20:43.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.398+0000 7f5f8d400640 1 -- start start 2024-04-15T15:20:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.399+0000 7f5f8d400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5f88162020 0x7f5f88162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.399+0000 7f5f8d400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5f880d1c90 0x7f5f880d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.399+0000 7f5f8d400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f880d0090 con 0x7f5f880cf110 2024-04-15T15:20:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.399+0000 7f5f8d400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5f88163440 con 0x7f5f880d1c90 2024-04-15T15:20:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.399+0000 7f5f8d400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f5f880b7b30 con 0x7f5f88162020 2024-04-15T15:20:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.400+0000 7f5f8ca00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5f880d1c90 0x7f5f880d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.400+0000 7f5f87400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5f880cf110 0x7f5f880d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60166/0 (socket says 172.21.15.49:60166) 2024-04-15T15:20:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.400+0000 7f5f87400640 1 -- 172.21.15.49:0/4141038734 learned_addr learned my addr 172.21.15.49:0/4141038734 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.400+0000 7f5f87e00640 1 --2- 172.21.15.49:0/4141038734 >> v2:172.21.15.107:3300/0 conn(0x7f5f88162020 0x7f5f88162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.400+0000 7f5f8ca00640 1 -- 172.21.15.49:0/4141038734 >> v2:172.21.15.107:3300/0 conn(0x7f5f88162020 msgr2=0x7f5f88162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.400+0000 7f5f8ca00640 1 --2- 172.21.15.49:0/4141038734 >> v2:172.21.15.107:3300/0 conn(0x7f5f88162020 0x7f5f88162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.400+0000 7f5f8ca00640 1 -- 172.21.15.49:0/4141038734 >> v1:172.21.15.49:6789/0 conn(0x7f5f880cf110 legacy=0x7f5f880d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.400+0000 7f5f8ca00640 1 -- 172.21.15.49:0/4141038734 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f880cfad0 con 0x7f5f880d1c90 2024-04-15T15:20:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.402+0000 7f5f8ca00640 1 --2- 172.21.15.49:0/4141038734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880d1c90 0x7f5f880d3390 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5f8808cd90 tx=0x7f5f8002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e96960dc652e7da server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.402+0000 7f5f86a00640 1 -- 172.21.15.49:0/4141038734 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80039070 con 0x7f5f880d1c90 2024-04-15T15:20:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.402+0000 7f5f86a00640 1 -- 172.21.15.49:0/4141038734 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f8002cc70 con 0x7f5f880d1c90 2024-04-15T15:20:43.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.402+0000 7f5f86a00640 1 -- 172.21.15.49:0/4141038734 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80034660 con 0x7f5f880d1c90 2024-04-15T15:20:43.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.403+0000 7f5f8d400640 1 -- 172.21.15.49:0/4141038734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880d1c90 msgr2=0x7f5f880d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.403+0000 7f5f8d400640 1 --2- 172.21.15.49:0/4141038734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880d1c90 0x7f5f880d3390 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5f8808cd90 tx=0x7f5f8002bec0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.403+0000 7f5f8d400640 1 -- 172.21.15.49:0/4141038734 shutdown_connections 2024-04-15T15:20:43.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.403+0000 7f5f8d400640 1 --2- 172.21.15.49:0/4141038734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880d1c90 0x7f5f880d3390 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.403+0000 7f5f8d400640 1 --2- 172.21.15.49:0/4141038734 >> v2:172.21.15.107:3300/0 conn(0x7f5f88162020 0x7f5f88162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.403+0000 7f5f8d400640 1 -- 172.21.15.49:0/4141038734 >> 172.21.15.49:0/4141038734 conn(0x7f5f8806cf00 msgr2=0x7f5f8806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:43.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.403+0000 7f5f8d400640 1 -- 172.21.15.49:0/4141038734 shutdown_connections 2024-04-15T15:20:43.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.403+0000 7f5f8d400640 1 -- 172.21.15.49:0/4141038734 wait complete. 2024-04-15T15:20:43.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.405+0000 7f5f8d400640 1 Processor -- start 2024-04-15T15:20:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.405+0000 7f5f8d400640 1 -- start start 2024-04-15T15:20:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.407+0000 7f5f8d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880cf110 0x7f5f8814ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.407+0000 7f5f8d400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f880d1c90 0x7f5f880cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.407+0000 7f5f8d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f88162020 0x7f5f880ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.407+0000 7f5f8d400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f880d2b90 con 0x7f5f880cf110 2024-04-15T15:20:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.407+0000 7f5f8d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5f880cf880 con 0x7f5f88162020 2024-04-15T15:20:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.407+0000 7f5f8d400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f5f880d2e70 con 0x7f5f880d1c90 2024-04-15T15:20:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.407+0000 7f5f87e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880cf110 0x7f5f8814ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.407+0000 7f5f87e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880cf110 0x7f5f8814ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41702/0 (socket says 172.21.15.49:41702) 2024-04-15T15:20:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.407+0000 7f5f87e00640 1 -- 172.21.15.49:0/3204266076 learned_addr learned my addr 172.21.15.49:0/3204266076 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f87e00640 1 -- 172.21.15.49:0/3204266076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f880d1c90 msgr2=0x7f5f880cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f87e00640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f880d1c90 0x7f5f880cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f87e00640 1 -- 172.21.15.49:0/3204266076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f88162020 msgr2=0x7f5f880ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f87e00640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f88162020 0x7f5f880ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f87e00640 1 -- 172.21.15.49:0/3204266076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f880cc750 con 0x7f5f880cf110 2024-04-15T15:20:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f87e00640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880cf110 0x7f5f8814ef80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800d800 tx=0x7f5f7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f7fe00640 1 -- 172.21.15.49:0/3204266076 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7800b6c0 con 0x7f5f880cf110 2024-04-15T15:20:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f7fe00640 1 -- 172.21.15.49:0/3204266076 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f78019070 con 0x7f5f880cf110 2024-04-15T15:20:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f8d400640 1 -- 172.21.15.49:0/3204266076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f88156950 con 0x7f5f880cf110 2024-04-15T15:20:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f7fe00640 1 -- 172.21.15.49:0/3204266076 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7800ba80 con 0x7f5f880cf110 2024-04-15T15:20:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.408+0000 7f5f8d400640 1 -- 172.21.15.49:0/3204266076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f88156ed0 con 0x7f5f880cf110 2024-04-15T15:20:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.409+0000 7f5f8d400640 1 -- 172.21.15.49:0/3204266076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f880755c0 con 0x7f5f880cf110 2024-04-15T15:20:43.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.411+0000 7f5f7fe00640 1 -- 172.21.15.49:0/3204266076 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5f7800bc20 con 0x7f5f880cf110 2024-04-15T15:20:43.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.412+0000 7f5f7fe00640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5f6806c980 0x7f5f6806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.412+0000 7f5f7fe00640 1 -- 172.21.15.49:0/3204266076 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f5f7808c0e0 con 0x7f5f880cf110 2024-04-15T15:20:43.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.412+0000 7f5f87400640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5f6806c980 0x7f5f6806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.413+0000 7f5f87400640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5f6806c980 0x7f5f6806ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5f74000f60 tx=0x7f5f74007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.420+0000 7f5f7fe00640 1 -- 172.21.15.49:0/3204266076 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f7805a4c0 con 0x7f5f880cf110 2024-04-15T15:20:43.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.624+0000 7f5f8d400640 1 -- 172.21.15.49:0/3204266076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f880b7650 con 0x7f5f880cf110 2024-04-15T15:20:43.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.626+0000 7f5f7fe00640 1 -- 172.21.15.49:0/3204266076 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16798 (secure 0 0 0) 0x7f5f78059dc0 con 0x7f5f880cf110 2024-04-15T15:20:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:43.629 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:28.618835+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6837/1023024717":"2024-04-16T15:20:28.618766+0000","172.21.15.49:6836/1023024717":"2024-04-16T15:20:28.618766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.628+0000 7f5f8d400640 1 -- 172.21.15.49:0/3204266076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5f6806c980 msgr2=0x7f5f6806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.628+0000 7f5f8d400640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5f6806c980 0x7f5f6806ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5f74000f60 tx=0x7f5f74007a80 comp rx=0 tx=0).stop 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.628+0000 7f5f8d400640 1 -- 172.21.15.49:0/3204266076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880cf110 msgr2=0x7f5f8814ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.628+0000 7f5f8d400640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880cf110 0x7f5f8814ef80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800d800 tx=0x7f5f7800dcc0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.629+0000 7f5f8d400640 1 -- 172.21.15.49:0/3204266076 shutdown_connections 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.629+0000 7f5f8d400640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f5f6806c980 0x7f5f6806ee40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.629+0000 7f5f8d400640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f880d1c90 0x7f5f880cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.629+0000 7f5f8d400640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f88162020 0x7f5f880ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.629+0000 7f5f8d400640 1 --2- 172.21.15.49:0/3204266076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f880cf110 0x7f5f8814ef80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.629+0000 7f5f8d400640 1 -- 172.21.15.49:0/3204266076 >> 172.21.15.49:0/3204266076 conn(0x7f5f8806cf00 msgr2=0x7f5f88074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:43.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.629+0000 7f5f8d400640 1 -- 172.21.15.49:0/3204266076 shutdown_connections 2024-04-15T15:20:43.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.629+0000 7f5f8d400640 1 -- 172.21.15.49:0/3204266076 wait complete. 2024-04-15T15:20:43.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:58.063498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:59.009342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:43.650 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:43.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.888+0000 7f0c43600640 1 Processor -- start 2024-04-15T15:20:43.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.888+0000 7f0c43600640 1 -- start start 2024-04-15T15:20:43.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.889+0000 7f0c43600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0c3c0b8b30 0x7f0c3c1696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.889+0000 7f0c43600640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c3c0d2c80 con 0x7f0c3c0d16f0 2024-04-15T15:20:43.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.889+0000 7f0c43600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c3c0d2f20 con 0x7f0c3c15eb50 2024-04-15T15:20:43.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.889+0000 7f0c43600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3c0d3850 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.889+0000 7f0c41600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0c3c0b8b30 0x7f0c3c1696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.889+0000 7f0c3be00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0c3c0d16f0 0x7f0c3c0d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60178/0 (socket says 172.21.15.49:60178) 2024-04-15T15:20:43.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.889+0000 7f0c3be00640 1 -- 172.21.15.49:0/1915969014 learned_addr learned my addr 172.21.15.49:0/1915969014 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:43.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.889+0000 7f0c41600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0c3c0b8b30 0x7f0c3c1696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41710/0 (socket says 172.21.15.49:41710) 2024-04-15T15:20:43.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.890+0000 7f0c41600640 1 -- 172.21.15.49:0/1915969014 >> v1:172.21.15.107:6789/0 conn(0x7f0c3c15eb50 legacy=0x7f0c3c0d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:43.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.890+0000 7f0c41600640 1 -- 172.21.15.49:0/1915969014 >> v1:172.21.15.49:6789/0 conn(0x7f0c3c0d16f0 legacy=0x7f0c3c0d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.890+0000 7f0c41600640 1 -- 172.21.15.49:0/1915969014 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c0d3200 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.890+0000 7f0c3b400640 1 -- 172.21.15.49:0/1915969014 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2513757128 0 0) 0x7f0c3c0d2c80 con 0x7f0c3c0d16f0 2024-04-15T15:20:43.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.891+0000 7f0c41600640 1 --2- 172.21.15.49:0/1915969014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 0x7f0c3c1696f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009d90 tx=0x7f0c3402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a07dd9f6ce4932 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.891+0000 7f0c3b400640 1 -- 172.21.15.49:0/1915969014 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34039070 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.891+0000 7f0c3b400640 1 -- 172.21.15.49:0/1915969014 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3402cc60 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.892+0000 7f0c3b400640 1 -- 172.21.15.49:0/1915969014 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34034660 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.892+0000 7f0c43600640 1 -- 172.21.15.49:0/1915969014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 msgr2=0x7f0c3c1696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.892+0000 7f0c43600640 1 --2- 172.21.15.49:0/1915969014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 0x7f0c3c1696f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009d90 tx=0x7f0c3402beb0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.892+0000 7f0c43600640 1 -- 172.21.15.49:0/1915969014 shutdown_connections 2024-04-15T15:20:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.892+0000 7f0c43600640 1 --2- 172.21.15.49:0/1915969014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 0x7f0c3c1696f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.892+0000 7f0c43600640 1 -- 172.21.15.49:0/1915969014 >> 172.21.15.49:0/1915969014 conn(0x7f0c3c06d1c0 msgr2=0x7f0c3c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.892+0000 7f0c43600640 1 -- 172.21.15.49:0/1915969014 shutdown_connections 2024-04-15T15:20:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.892+0000 7f0c43600640 1 -- 172.21.15.49:0/1915969014 wait complete. 2024-04-15T15:20:43.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.894+0000 7f0c43600640 1 Processor -- start 2024-04-15T15:20:43.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.894+0000 7f0c43600640 1 -- start start 2024-04-15T15:20:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.895+0000 7f0c43600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 0x7f0c3c14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.895+0000 7f0c43600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d16f0 0x7f0c3c0b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.895+0000 7f0c43600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c15eb50 0x7f0c3c1610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.895+0000 7f0c43600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c0cfed0 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.895+0000 7f0c43600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c15ef30 con 0x7f0c3c15eb50 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.895+0000 7f0c43600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3c0d4520 con 0x7f0c3c0d16f0 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.895+0000 7f0c40c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 0x7f0c3c14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.895+0000 7f0c40c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 0x7f0c3c14fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41718/0 (socket says 172.21.15.49:41718) 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.896+0000 7f0c40c00640 1 -- 172.21.15.49:0/3399717670 learned_addr learned my addr 172.21.15.49:0/3399717670 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.896+0000 7f0c41600640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c15eb50 0x7f0c3c1610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.896+0000 7f0c40c00640 1 -- 172.21.15.49:0/3399717670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d16f0 msgr2=0x7f0c3c0b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.896+0000 7f0c3be00640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d16f0 0x7f0c3c0b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.896+0000 7f0c40c00640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d16f0 0x7f0c3c0b6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.896+0000 7f0c40c00640 1 -- 172.21.15.49:0/3399717670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c15eb50 msgr2=0x7f0c3c1610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.896+0000 7f0c40c00640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c15eb50 0x7f0c3c1610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.896+0000 7f0c40c00640 1 -- 172.21.15.49:0/3399717670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c0b64e0 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.896+0000 7f0c41600640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c15eb50 0x7f0c3c1610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.896+0000 7f0c40c00640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 0x7f0c3c14fc60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00b3f0 tx=0x7f0c2c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.897+0000 7f0c38c00640 1 -- 172.21.15.49:0/3399717670 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c015020 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.897+0000 7f0c38c00640 1 -- 172.21.15.49:0/3399717670 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2c011830 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.897+0000 7f0c43600640 1 -- 172.21.15.49:0/3399717670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3c0b5960 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.897+0000 7f0c38c00640 1 -- 172.21.15.49:0/3399717670 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c011c20 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.897+0000 7f0c43600640 1 -- 172.21.15.49:0/3399717670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3c0b5f20 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.898+0000 7f0c43600640 1 -- 172.21.15.49:0/3399717670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c3c068c80 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.899+0000 7f0c38c00640 1 -- 172.21.15.49:0/3399717670 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0c2c018020 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.900+0000 7f0c38c00640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0c1806c980 0x7f0c1806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:43.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.900+0000 7f0c38c00640 1 -- 172.21.15.49:0/3399717670 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f0c2c08b910 con 0x7f0c3c0b8b30 2024-04-15T15:20:43.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.900+0000 7f0c3be00640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0c1806c980 0x7f0c1806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:43.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.901+0000 7f0c3be00640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0c1806c980 0x7f0c1806ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c1f7a40 tx=0x7f0c24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:43.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:43.904+0000 7f0c38c00640 1 -- 172.21.15.49:0/3399717670 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c2c01f030 con 0x7f0c3c0b8b30 2024-04-15T15:20:44.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.110+0000 7f0c43600640 1 -- 172.21.15.49:0/3399717670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c3c0cb9f0 con 0x7f0c3c0b8b30 2024-04-15T15:20:44.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.111+0000 7f0c38c00640 1 -- 172.21.15.49:0/3399717670 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f0c3c0cb9f0 con 0x7f0c3c0b8b30 2024-04-15T15:20:44.113 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:20:44.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.114+0000 7f0c43600640 1 -- 172.21.15.49:0/3399717670 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0c1806c980 msgr2=0x7f0c1806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.114+0000 7f0c43600640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0c1806c980 0x7f0c1806ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c1f7a40 tx=0x7f0c24009040 comp rx=0 tx=0).stop 2024-04-15T15:20:44.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.114+0000 7f0c43600640 1 -- 172.21.15.49:0/3399717670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 msgr2=0x7f0c3c14fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.114+0000 7f0c43600640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 0x7f0c3c14fc60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00b3f0 tx=0x7f0c2c00b8b0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.114+0000 7f0c43600640 1 -- 172.21.15.49:0/3399717670 shutdown_connections 2024-04-15T15:20:44.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.114+0000 7f0c43600640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f0c1806c980 0x7f0c1806ee40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.114+0000 7f0c43600640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d16f0 0x7f0c3c0b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.114+0000 7f0c43600640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c15eb50 0x7f0c3c1610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.114+0000 7f0c43600640 1 --2- 172.21.15.49:0/3399717670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c3c0b8b30 0x7f0c3c14fc60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.114+0000 7f0c43600640 1 -- 172.21.15.49:0/3399717670 >> 172.21.15.49:0/3399717670 conn(0x7f0c3c06d1c0 msgr2=0x7f0c3c0b9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:44.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.115+0000 7f0c43600640 1 -- 172.21.15.49:0/3399717670 shutdown_connections 2024-04-15T15:20:44.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.115+0000 7f0c43600640 1 -- 172.21.15.49:0/3399717670 wait complete. 2024-04-15T15:20:44.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:44.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.372+0000 7f4742000640 1 Processor -- start 2024-04-15T15:20:44.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.372+0000 7f4742000640 1 -- start start 2024-04-15T15:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.373+0000 7f4742000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c162730 0x7f473c0b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.373+0000 7f4742000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f473c162dd0 con 0x7f473c15c680 2024-04-15T15:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.373+0000 7f4742000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f473c1630b0 con 0x7f473c0d2b50 2024-04-15T15:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.373+0000 7f4742000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f473c162b90 con 0x7f473c162730 2024-04-15T15:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.374+0000 7f4740c00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f473c15c680 0x7f473c15f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56534/0 (socket says 172.21.15.49:56534) 2024-04-15T15:20:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.374+0000 7f4740c00640 1 -- 172.21.15.49:0/4198641639 learned_addr learned my addr 172.21.15.49:0/4198641639 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.374+0000 7f4733e00640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c162730 0x7f473c0b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.374+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593278480 0 0) 0x7f473c162dd0 con 0x7f473c15c680 2024-04-15T15:20:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.374+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4720003520 con 0x7f473c15c680 2024-04-15T15:20:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3239655119 0 0) 0x7f4720003520 con 0x7f473c15c680 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c162730 msgr2=0x7f473c0b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c162730 0x7f473c0b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 >> v1:172.21.15.107:6789/0 conn(0x7f473c0d2b50 legacy=0x7f473c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c163390 con 0x7f473c15c680 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f472c001670 con 0x7f473c15c680 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 >> v1:172.21.15.49:6789/0 conn(0x7f473c15c680 legacy=0x7f473c15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47200039e0 0x7f4720003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720004800 0x7f4720004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c162730 0x7f47200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f473c162b90 con 0x7f4720004800 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f472000baa0 con 0x7f47200039e0 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f472000bca0 con 0x7f473c162730 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.375+0000 7f4733e00640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720004800 0x7f4720004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.376+0000 7f4740c00640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47200039e0 0x7f4720003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.376+0000 7f4733e00640 1 -- 172.21.15.49:0/4198641639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c162730 msgr2=0x7f47200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.376+0000 7f4733e00640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c162730 0x7f47200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.376+0000 7f4733e00640 1 -- 172.21.15.49:0/4198641639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47200039e0 msgr2=0x7f4720003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.376+0000 7f4733e00640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47200039e0 0x7f4720003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.376+0000 7f4733e00640 1 -- 172.21.15.49:0/4198641639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4720003520 con 0x7f4720004800 2024-04-15T15:20:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.377+0000 7f4733e00640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720004800 0x7f4720004c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4724009a90 tx=0x7f4720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198a42b5221a84d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.377+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724039070 con 0x7f4720004800 2024-04-15T15:20:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.377+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f472402c6b0 con 0x7f4720004800 2024-04-15T15:20:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.377+0000 7f473be00640 1 -- 172.21.15.49:0/4198641639 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724009690 con 0x7f4720004800 2024-04-15T15:20:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.378+0000 7f4742000640 1 -- 172.21.15.49:0/4198641639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720004800 msgr2=0x7f4720004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.378+0000 7f4742000640 1 --2- 172.21.15.49:0/4198641639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4720004800 0x7f4720004c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4724009a90 tx=0x7f4720005830 comp rx=0 tx=0).stop 2024-04-15T15:20:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.378+0000 7f4741600640 1 -- 172.21.15.49:0/4198641639 reap_dead start 2024-04-15T15:20:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.378+0000 7f4742000640 1 -- 172.21.15.49:0/4198641639 shutdown_connections 2024-04-15T15:20:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.378+0000 7f4742000640 1 -- 172.21.15.49:0/4198641639 >> 172.21.15.49:0/4198641639 conn(0x7f473c073480 msgr2=0x7f473c073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.378+0000 7f4742000640 1 -- 172.21.15.49:0/4198641639 shutdown_connections 2024-04-15T15:20:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.378+0000 7f4742000640 1 -- 172.21.15.49:0/4198641639 wait complete. 2024-04-15T15:20:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.380+0000 7f4742000640 1 Processor -- start 2024-04-15T15:20:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.380+0000 7f4742000640 1 -- start start 2024-04-15T15:20:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.381+0000 7f4742000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4720004800 0x7f473c0ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.381+0000 7f4742000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c0cbfa0 0x7f473c0d49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.381+0000 7f4742000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f473c151840 0x7f473c151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.381+0000 7f4742000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f473c15ff60 con 0x7f473c151840 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.381+0000 7f4742000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f473c15ca60 con 0x7f4720004800 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.381+0000 7f4742000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f473c0d1e80 con 0x7f473c0cbfa0 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.381+0000 7f4741600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f473c151840 0x7f473c151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.382+0000 7f4741600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f473c151840 0x7f473c151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56648/0 (socket says 172.21.15.49:56648) 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.382+0000 7f4741600640 1 -- 172.21.15.49:0/4046815799 learned_addr learned my addr 172.21.15.49:0/4046815799 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.382+0000 7f4740c00640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4720004800 0x7f473c0ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.382+0000 7f4733e00640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c0cbfa0 0x7f473c0d49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.382+0000 7f4741600640 1 -- 172.21.15.49:0/4046815799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c0cbfa0 msgr2=0x7f473c0d49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.382+0000 7f4741600640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c0cbfa0 0x7f473c0d49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.382+0000 7f4741600640 1 -- 172.21.15.49:0/4046815799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4720004800 msgr2=0x7f473c0ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.382+0000 7f4741600640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4720004800 0x7f473c0ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.382+0000 7f4741600640 1 -- 172.21.15.49:0/4046815799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c0cb9e0 con 0x7f473c151840 2024-04-15T15:20:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.382+0000 7f4741600640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f473c151840 0x7f473c151ca0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f473c0d5880 tx=0x7f473400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.383+0000 7f4739600640 1 -- 172.21.15.49:0/4046815799 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734017070 con 0x7f473c151840 2024-04-15T15:20:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.383+0000 7f4739600640 1 -- 172.21.15.49:0/4046815799 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4734005d60 con 0x7f473c151840 2024-04-15T15:20:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.383+0000 7f4739600640 1 -- 172.21.15.49:0/4046815799 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473400a850 con 0x7f473c151840 2024-04-15T15:20:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.383+0000 7f4742000640 1 -- 172.21.15.49:0/4046815799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c0cbcc0 con 0x7f473c151840 2024-04-15T15:20:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.383+0000 7f4742000640 1 -- 172.21.15.49:0/4046815799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c0c9b10 con 0x7f473c151840 2024-04-15T15:20:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.384+0000 7f4742000640 1 -- 172.21.15.49:0/4046815799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473c0c9d10 con 0x7f473c151840 2024-04-15T15:20:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.385+0000 7f4739600640 1 -- 172.21.15.49:0/4046815799 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f473400a9f0 con 0x7f473c151840 2024-04-15T15:20:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.386+0000 7f4739600640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f471806c980 0x7f471806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.386+0000 7f4739600640 1 -- 172.21.15.49:0/4046815799 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f473408add0 con 0x7f473c151840 2024-04-15T15:20:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.387+0000 7f4740c00640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f471806c980 0x7f471806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.387+0000 7f4740c00640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f471806c980 0x7f471806ee40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f472c001050 tx=0x7f472c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.391+0000 7f4739600640 1 -- 172.21.15.49:0/4046815799 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f473c0c9d10 con 0x7f473c151840 2024-04-15T15:20:44.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.596+0000 7f4742000640 1 -- 172.21.15.49:0/4046815799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f473c0755c0 con 0x7f473c151840 2024-04-15T15:20:44.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.597+0000 7f4739600640 1 -- 172.21.15.49:0/4046815799 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7f4734059240 con 0x7f473c151840 2024-04-15T15:20:44.598 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:20:44.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 -- 172.21.15.49:0/4046815799 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f471806c980 msgr2=0x7f471806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f471806c980 0x7f471806ee40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f472c001050 tx=0x7f472c009040 comp rx=0 tx=0).stop 2024-04-15T15:20:44.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 -- 172.21.15.49:0/4046815799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f473c151840 msgr2=0x7f473c151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f473c151840 0x7f473c151ca0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f473c0d5880 tx=0x7f473400cb90 comp rx=0 tx=0).stop 2024-04-15T15:20:44.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 -- 172.21.15.49:0/4046815799 shutdown_connections 2024-04-15T15:20:44.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f471806c980 0x7f471806ee40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f473c0cbfa0 0x7f473c0d49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f473c151840 0x7f473c151ca0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 --2- 172.21.15.49:0/4046815799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4720004800 0x7f473c0ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 -- 172.21.15.49:0/4046815799 >> 172.21.15.49:0/4046815799 conn(0x7f473c073480 msgr2=0x7f473c0ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:44.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 -- 172.21.15.49:0/4046815799 shutdown_connections 2024-04-15T15:20:44.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.599+0000 7f4742000640 1 -- 172.21.15.49:0/4046815799 wait complete. 2024-04-15T15:20:44.623 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.850+0000 7f3f03200640 1 Processor -- start 2024-04-15T15:20:44.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.851+0000 7f3f03200640 1 -- start start 2024-04-15T15:20:44.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.852+0000 7f3f03200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3efc0b4390 0x7f3efc0b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.852+0000 7f3f03200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3efc162f60 0x7f3efc06f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.852+0000 7f3f03200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3efc072810 con 0x7f3efc162b80 2024-04-15T15:20:44.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.852+0000 7f3f03200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3efc0725f0 con 0x7f3efc162f60 2024-04-15T15:20:44.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.852+0000 7f3f03200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f3efc0b49c0 con 0x7f3efc0b4390 2024-04-15T15:20:44.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.853+0000 7f3f02800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3efc162f60 0x7f3efc06f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.853+0000 7f3f02800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3efc162f60 0x7f3efc06f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56662/0 (socket says 172.21.15.49:56662) 2024-04-15T15:20:44.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.853+0000 7f3f02800640 1 -- 172.21.15.49:0/657882978 learned_addr learned my addr 172.21.15.49:0/657882978 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:44.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.853+0000 7f3f01400640 1 --2- 172.21.15.49:0/657882978 >> v2:172.21.15.107:3300/0 conn(0x7f3efc0b4390 0x7f3efc0b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.854+0000 7f3f02800640 1 -- 172.21.15.49:0/657882978 >> v2:172.21.15.107:3300/0 conn(0x7f3efc0b4390 msgr2=0x7f3efc0b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:44.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.854+0000 7f3f02800640 1 --2- 172.21.15.49:0/657882978 >> v2:172.21.15.107:3300/0 conn(0x7f3efc0b4390 0x7f3efc0b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.854+0000 7f3f02800640 1 -- 172.21.15.49:0/657882978 >> v1:172.21.15.107:6789/0 conn(0x7f3efc162b80 legacy=0x7f3efc071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.854+0000 7f3f02800640 1 -- 172.21.15.49:0/657882978 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc0b4ca0 con 0x7f3efc162f60 2024-04-15T15:20:44.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.855+0000 7f3f02800640 1 --2- 172.21.15.49:0/657882978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc162f60 0x7f3efc06f210 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0b3430 tx=0x7f3ef802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626bbe4d9588581a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.855+0000 7f3f00a00640 1 -- 172.21.15.49:0/657882978 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef802cb10 con 0x7f3efc162f60 2024-04-15T15:20:44.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.855+0000 7f3f00a00640 1 -- 172.21.15.49:0/657882978 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef802ccb0 con 0x7f3efc162f60 2024-04-15T15:20:44.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.855+0000 7f3f00a00640 1 -- 172.21.15.49:0/657882978 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef802ce60 con 0x7f3efc162f60 2024-04-15T15:20:44.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.856+0000 7f3f03200640 1 -- 172.21.15.49:0/657882978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc162f60 msgr2=0x7f3efc06f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.856+0000 7f3f03200640 1 --2- 172.21.15.49:0/657882978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc162f60 0x7f3efc06f210 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0b3430 tx=0x7f3ef802bf70 comp rx=0 tx=0).stop 2024-04-15T15:20:44.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.856+0000 7f3f03200640 1 -- 172.21.15.49:0/657882978 shutdown_connections 2024-04-15T15:20:44.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.856+0000 7f3f03200640 1 --2- 172.21.15.49:0/657882978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc162f60 0x7f3efc06f210 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.856+0000 7f3f03200640 1 --2- 172.21.15.49:0/657882978 >> v2:172.21.15.107:3300/0 conn(0x7f3efc0b4390 0x7f3efc0b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.856+0000 7f3f03200640 1 -- 172.21.15.49:0/657882978 >> 172.21.15.49:0/657882978 conn(0x7f3efc06b930 msgr2=0x7f3efc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:44.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.856+0000 7f3f03200640 1 -- 172.21.15.49:0/657882978 shutdown_connections 2024-04-15T15:20:44.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.856+0000 7f3f03200640 1 -- 172.21.15.49:0/657882978 wait complete. 2024-04-15T15:20:44.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.858+0000 7f3f03200640 1 Processor -- start 2024-04-15T15:20:44.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.858+0000 7f3f03200640 1 -- start start 2024-04-15T15:20:44.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.859+0000 7f3f03200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc0b4390 0x7f3efc15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.859+0000 7f3f03200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3efc162b80 0x7f3efc0ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.859+0000 7f3f03200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3efc162f60 0x7f3efc165990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.859+0000 7f3f03200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc06f750 con 0x7f3efc0b4390 2024-04-15T15:20:44.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.859+0000 7f3f03200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc0b5260 con 0x7f3efc162f60 2024-04-15T15:20:44.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.859+0000 7f3f03200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f3efc06f8d0 con 0x7f3efc162b80 2024-04-15T15:20:44.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.859+0000 7f3f01e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc0b4390 0x7f3efc15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f01e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc0b4390 0x7f3efc15ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56664/0 (socket says 172.21.15.49:56664) 2024-04-15T15:20:44.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f01e00640 1 -- 172.21.15.49:0/2200496485 learned_addr learned my addr 172.21.15.49:0/2200496485 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f01400640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3efc162b80 0x7f3efc0ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f02800640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3efc162f60 0x7f3efc165990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f01e00640 1 -- 172.21.15.49:0/2200496485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3efc162b80 msgr2=0x7f3efc0ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f01e00640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3efc162b80 0x7f3efc0ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f01e00640 1 -- 172.21.15.49:0/2200496485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3efc162f60 msgr2=0x7f3efc165990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f01e00640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3efc162f60 0x7f3efc165990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f01e00640 1 -- 172.21.15.49:0/2200496485 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc159bf0 con 0x7f3efc0b4390 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f02800640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3efc162f60 0x7f3efc165990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.860+0000 7f3f01e00640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc0b4390 0x7f3efc15ac50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c400 tx=0x7f3ef000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.861+0000 7f3ef6000640 1 -- 172.21.15.49:0/2200496485 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00052d0 con 0x7f3efc0b4390 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.861+0000 7f3ef6000640 1 -- 172.21.15.49:0/2200496485 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef0005490 con 0x7f3efc0b4390 2024-04-15T15:20:44.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.861+0000 7f3f03200640 1 -- 172.21.15.49:0/2200496485 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc159e90 con 0x7f3efc0b4390 2024-04-15T15:20:44.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.861+0000 7f3ef6000640 1 -- 172.21.15.49:0/2200496485 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0007260 con 0x7f3efc0b4390 2024-04-15T15:20:44.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.861+0000 7f3f03200640 1 -- 172.21.15.49:0/2200496485 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc15a450 con 0x7f3efc0b4390 2024-04-15T15:20:44.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.862+0000 7f3f03200640 1 -- 172.21.15.49:0/2200496485 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc0755c0 con 0x7f3efc0b4390 2024-04-15T15:20:44.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.863+0000 7f3ef6000640 1 -- 172.21.15.49:0/2200496485 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3ef0007400 con 0x7f3efc0b4390 2024-04-15T15:20:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.864+0000 7f3ef6000640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3ee006c980 0x7f3ee006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.864+0000 7f3ef6000640 1 -- 172.21.15.49:0/2200496485 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f3ef008b1f0 con 0x7f3efc0b4390 2024-04-15T15:20:44.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.865+0000 7f3f01400640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3ee006c980 0x7f3ee006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:44.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.865+0000 7f3f01400640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3ee006c980 0x7f3ee006ee40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3efc1fbf00 tx=0x7f3eec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:44.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:44.869+0000 7f3ef6000640 1 -- 172.21.15.49:0/2200496485 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ef0059560 con 0x7f3efc0b4390 2024-04-15T15:20:45.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.075+0000 7f3f03200640 1 -- 172.21.15.49:0/2200496485 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3efc0720f0 con 0x7f3efc0b4390 2024-04-15T15:20:45.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.077+0000 7f3ef6000640 1 -- 172.21.15.49:0/2200496485 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f3ef0058e60 con 0x7f3efc0b4390 2024-04-15T15:20:45.078 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:20:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.079+0000 7f3f03200640 1 -- 172.21.15.49:0/2200496485 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3ee006c980 msgr2=0x7f3ee006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.079+0000 7f3f03200640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3ee006c980 0x7f3ee006ee40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3efc1fbf00 tx=0x7f3eec008040 comp rx=0 tx=0).stop 2024-04-15T15:20:45.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.080+0000 7f3f03200640 1 -- 172.21.15.49:0/2200496485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc0b4390 msgr2=0x7f3efc15ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.080+0000 7f3f03200640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc0b4390 0x7f3efc15ac50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c400 tx=0x7f3ef000c8c0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.080+0000 7f3f03200640 1 -- 172.21.15.49:0/2200496485 shutdown_connections 2024-04-15T15:20:45.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.080+0000 7f3f03200640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f3ee006c980 0x7f3ee006ee40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.080+0000 7f3f03200640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3efc162b80 0x7f3efc0ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.080+0000 7f3f03200640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3efc162f60 0x7f3efc165990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.080+0000 7f3f03200640 1 --2- 172.21.15.49:0/2200496485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3efc0b4390 0x7f3efc15ac50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.080+0000 7f3f03200640 1 -- 172.21.15.49:0/2200496485 >> 172.21.15.49:0/2200496485 conn(0x7f3efc06b930 msgr2=0x7f3efc0b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.080+0000 7f3f03200640 1 -- 172.21.15.49:0/2200496485 shutdown_connections 2024-04-15T15:20:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.080+0000 7f3f03200640 1 -- 172.21.15.49:0/2200496485 wait complete. 2024-04-15T15:20:45.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:20:45.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.332+0000 7f9074c00640 1 Processor -- start 2024-04-15T15:20:45.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.333+0000 7f9074c00640 1 -- start start 2024-04-15T15:20:45.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.334+0000 7f9074c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90700d1550 0x7f90700d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.334+0000 7f9074c00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f90700d32e0 0x7f90700d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.334+0000 7f9074c00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90700b78f0 con 0x7f907015eab0 2024-04-15T15:20:45.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.334+0000 7f9074c00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f90700d1ef0 con 0x7f90700d1550 2024-04-15T15:20:45.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.334+0000 7f9074c00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f90700d2e50 con 0x7f90700d32e0 2024-04-15T15:20:45.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.335+0000 7f9067e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90700d1550 0x7f90700d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.335+0000 7f9067e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90700d1550 0x7f90700d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56670/0 (socket says 172.21.15.49:56670) 2024-04-15T15:20:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.335+0000 7f9067e00640 1 -- 172.21.15.49:0/2916106173 learned_addr learned my addr 172.21.15.49:0/2916106173 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.335+0000 7f906fe00640 1 --2- 172.21.15.49:0/2916106173 >> v2:172.21.15.107:3300/0 conn(0x7f90700d32e0 0x7f90700d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.335+0000 7f9067e00640 1 -- 172.21.15.49:0/2916106173 >> v2:172.21.15.107:3300/0 conn(0x7f90700d32e0 msgr2=0x7f90700d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.335+0000 7f9067e00640 1 --2- 172.21.15.49:0/2916106173 >> v2:172.21.15.107:3300/0 conn(0x7f90700d32e0 0x7f90700d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.335+0000 7f9067e00640 1 -- 172.21.15.49:0/2916106173 >> v1:172.21.15.107:6789/0 conn(0x7f907015eab0 legacy=0x7f9070162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.335+0000 7f9067e00640 1 -- 172.21.15.49:0/2916106173 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9070162d40 con 0x7f90700d1550 2024-04-15T15:20:45.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.336+0000 7f9067e00640 1 --2- 172.21.15.49:0/2916106173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 0x7f90700d19b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f907008d710 tx=0x7f905802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfaf1cef448296c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.337+0000 7f906ea00640 1 -- 172.21.15.49:0/2916106173 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058035020 con 0x7f90700d1550 2024-04-15T15:20:45.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.337+0000 7f906ea00640 1 -- 172.21.15.49:0/2916106173 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9058031870 con 0x7f90700d1550 2024-04-15T15:20:45.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.337+0000 7f906ea00640 1 -- 172.21.15.49:0/2916106173 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058031c40 con 0x7f90700d1550 2024-04-15T15:20:45.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.337+0000 7f9074c00640 1 -- 172.21.15.49:0/2916106173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 msgr2=0x7f90700d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.337+0000 7f9074c00640 1 --2- 172.21.15.49:0/2916106173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 0x7f90700d19b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f907008d710 tx=0x7f905802b910 comp rx=0 tx=0).stop 2024-04-15T15:20:45.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.338+0000 7f9074c00640 1 -- 172.21.15.49:0/2916106173 shutdown_connections 2024-04-15T15:20:45.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.338+0000 7f9074c00640 1 --2- 172.21.15.49:0/2916106173 >> v2:172.21.15.107:3300/0 conn(0x7f90700d32e0 0x7f90700d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.338+0000 7f9074c00640 1 --2- 172.21.15.49:0/2916106173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 0x7f90700d19b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.338+0000 7f9074c00640 1 -- 172.21.15.49:0/2916106173 >> 172.21.15.49:0/2916106173 conn(0x7f9070073240 msgr2=0x7f9070073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:45.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.338+0000 7f9074c00640 1 -- 172.21.15.49:0/2916106173 shutdown_connections 2024-04-15T15:20:45.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.338+0000 7f9074c00640 1 -- 172.21.15.49:0/2916106173 wait complete. 2024-04-15T15:20:45.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.339+0000 7f9074c00640 1 Processor -- start 2024-04-15T15:20:45.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.340+0000 7f9074c00640 1 -- start start 2024-04-15T15:20:45.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f9074c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 0x7f90700ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f9074c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90700d32e0 0x7f90700cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f9074c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907015eab0 0x7f90701e82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f9074c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90700d2ae0 con 0x7f90700d1550 2024-04-15T15:20:45.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f9074c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90701632f0 con 0x7f907015eab0 2024-04-15T15:20:45.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f9074c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f907015ee90 con 0x7f90700d32e0 2024-04-15T15:20:45.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f906f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 0x7f90700ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f906f400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 0x7f90700ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56680/0 (socket says 172.21.15.49:56680) 2024-04-15T15:20:45.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f906f400640 1 -- 172.21.15.49:0/3139859120 learned_addr learned my addr 172.21.15.49:0/3139859120 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:45.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f9067e00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90700d32e0 0x7f90700cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f906fe00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907015eab0 0x7f90701e82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f906f400640 1 -- 172.21.15.49:0/3139859120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90700d32e0 msgr2=0x7f90700cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f906f400640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90700d32e0 0x7f90700cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f906f400640 1 -- 172.21.15.49:0/3139859120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907015eab0 msgr2=0x7f90701e82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f906f400640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907015eab0 0x7f90701e82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.341+0000 7f906f400640 1 -- 172.21.15.49:0/3139859120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90700cdcc0 con 0x7f90700d1550 2024-04-15T15:20:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.342+0000 7f9067e00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90700d32e0 0x7f90700cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.342+0000 7f906fe00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907015eab0 0x7f90701e82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.342+0000 7f906f400640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 0x7f90700ccd20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f906000b3f0 tx=0x7f906000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.342+0000 7f9067400640 1 -- 172.21.15.49:0/3139859120 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060015020 con 0x7f90700d1550 2024-04-15T15:20:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.342+0000 7f9067400640 1 -- 172.21.15.49:0/3139859120 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9060011830 con 0x7f90700d1550 2024-04-15T15:20:45.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.342+0000 7f9067400640 1 -- 172.21.15.49:0/3139859120 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060011c20 con 0x7f90700d1550 2024-04-15T15:20:45.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.342+0000 7f9074c00640 1 -- 172.21.15.49:0/3139859120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90700cdfa0 con 0x7f90700d1550 2024-04-15T15:20:45.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.342+0000 7f9074c00640 1 -- 172.21.15.49:0/3139859120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90700ce280 con 0x7f90700d1550 2024-04-15T15:20:45.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.343+0000 7f9074c00640 1 -- 172.21.15.49:0/3139859120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90700755c0 con 0x7f90700d1550 2024-04-15T15:20:45.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.344+0000 7f9067400640 1 -- 172.21.15.49:0/3139859120 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9060018020 con 0x7f90700d1550 2024-04-15T15:20:45.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.345+0000 7f9067400640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f904c06c980 0x7f904c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.345+0000 7f9067400640 1 -- 172.21.15.49:0/3139859120 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f906008b8a0 con 0x7f90700d1550 2024-04-15T15:20:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.346+0000 7f9067e00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f904c06c980 0x7f904c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.347+0000 7f9067e00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f904c06c980 0x7f904c06ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9058036000 tx=0x7f9058008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:45.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.350+0000 7f9067400640 1 -- 172.21.15.49:0/3139859120 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f906001f030 con 0x7f90700d1550 2024-04-15T15:20:45.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.567+0000 7f9074c00640 1 -- 172.21.15.49:0/3139859120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f90700d2460 con 0x7f90700d1550 2024-04-15T15:20:45.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.567+0000 7f9067400640 1 -- 172.21.15.49:0/3139859120 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) ==== 80+0+122 (secure 0 0 0) 0x7f90600172c0 con 0x7f90700d1550 2024-04-15T15:20:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6837/1023024717 2024-04-16T15:20:28.618766+0000 2024-04-15T15:20:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6836/1023024717 2024-04-16T15:20:28.618766+0000 2024-04-15T15:20:45.570 INFO:teuthology.orchestra.run.smithi049.stderr:listed 2 entries 2024-04-15T15:20:45.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 -- 172.21.15.49:0/3139859120 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f904c06c980 msgr2=0x7f904c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f904c06c980 0x7f904c06ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9058036000 tx=0x7f9058008030 comp rx=0 tx=0).stop 2024-04-15T15:20:45.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 -- 172.21.15.49:0/3139859120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 msgr2=0x7f90700ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 0x7f90700ccd20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f906000b3f0 tx=0x7f906000b8b0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 -- 172.21.15.49:0/3139859120 shutdown_connections 2024-04-15T15:20:45.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f904c06c980 0x7f904c06ee40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90700d32e0 0x7f90700cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907015eab0 0x7f90701e82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 --2- 172.21.15.49:0/3139859120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90700d1550 0x7f90700ccd20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:45.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 -- 172.21.15.49:0/3139859120 >> 172.21.15.49:0/3139859120 conn(0x7f9070073240 msgr2=0x7f90700d47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:45.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.570+0000 7f9074c00640 1 -- 172.21.15.49:0/3139859120 shutdown_connections 2024-04-15T15:20:45.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.571+0000 7f9074c00640 1 -- 172.21.15.49:0/3139859120 wait complete. 2024-04-15T15:20:45.593 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-15T15:20:45.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:45.593 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.3 2024-04-15T15:20:45.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:20:45.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:45.711 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-15T15:20:45.762 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:45.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.994+0000 7fdd04a00640 1 Processor -- start 2024-04-15T15:20:45.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.994+0000 7fdd04a00640 1 -- start start 2024-04-15T15:20:45.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.996+0000 7fdd04a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdd00162c80 0x7fdd000b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:45.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.996+0000 7fdd04a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd00163060 con 0x7fdd00071f50 2024-04-15T15:20:45.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.996+0000 7fdd04a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd000afae0 con 0x7fdd000afd00 2024-04-15T15:20:45.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.996+0000 7fdd04a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fdd000b4ad0 con 0x7fdd00162c80 2024-04-15T15:20:45.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.996+0000 7fdcfe800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdd00162c80 0x7fdd000b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.996+0000 7fdcff200640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdd00071f50 0x7fdd000b4f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56546/0 (socket says 172.21.15.49:56546) 2024-04-15T15:20:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.996+0000 7fdcfe800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdd00162c80 0x7fdd000b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56694/0 (socket says 172.21.15.49:56694) 2024-04-15T15:20:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.997+0000 7fdcff200640 1 -- 172.21.15.49:0/2901697376 learned_addr learned my addr 172.21.15.49:0/2901697376 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.997+0000 7fdcfe800640 1 -- 172.21.15.49:0/2901697376 >> v1:172.21.15.107:6789/0 conn(0x7fdd000afd00 legacy=0x7fdd000ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.997+0000 7fdcfe800640 1 -- 172.21.15.49:0/2901697376 >> v1:172.21.15.49:6789/0 conn(0x7fdd00071f50 legacy=0x7fdd000b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.997+0000 7fdcfe800640 1 -- 172.21.15.49:0/2901697376 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd000b3c60 con 0x7fdd00162c80 2024-04-15T15:20:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.997+0000 7fdcfd400640 1 -- 172.21.15.49:0/2901697376 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407229527 0 0) 0x7fdcf8001490 con 0x7fdd00071f50 2024-04-15T15:20:46.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.998+0000 7fdcfe800640 1 --2- 172.21.15.49:0/2901697376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 0x7fdd000b3680 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00097a0 tx=0x7fdcf002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec58bf00bb683c36 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.999+0000 7fdcfd400640 1 -- 172.21.15.49:0/2901697376 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0035020 con 0x7fdd00162c80 2024-04-15T15:20:46.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.999+0000 7fdcfd400640 1 -- 172.21.15.49:0/2901697376 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf0031830 con 0x7fdd00162c80 2024-04-15T15:20:46.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.999+0000 7fdcfd400640 1 -- 172.21.15.49:0/2901697376 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0031c00 con 0x7fdd00162c80 2024-04-15T15:20:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.999+0000 7fdd04a00640 1 -- 172.21.15.49:0/2901697376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 msgr2=0x7fdd000b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.999+0000 7fdd04a00640 1 --2- 172.21.15.49:0/2901697376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 0x7fdd000b3680 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00097a0 tx=0x7fdcf002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.999+0000 7fdd04a00640 1 -- 172.21.15.49:0/2901697376 shutdown_connections 2024-04-15T15:20:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.999+0000 7fdd04a00640 1 --2- 172.21.15.49:0/2901697376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 0x7fdd000b3680 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:45.999+0000 7fdd04a00640 1 -- 172.21.15.49:0/2901697376 >> 172.21.15.49:0/2901697376 conn(0x7fdd0006b930 msgr2=0x7fdd0006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.000+0000 7fdd04a00640 1 -- 172.21.15.49:0/2901697376 shutdown_connections 2024-04-15T15:20:46.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.000+0000 7fdd04a00640 1 -- 172.21.15.49:0/2901697376 wait complete. 2024-04-15T15:20:46.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.001+0000 7fdd04a00640 1 Processor -- start 2024-04-15T15:20:46.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.002+0000 7fdd04a00640 1 -- start start 2024-04-15T15:20:46.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdd04a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd00071f50 0x7fdd001ed470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdd04a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd000afd00 0x7fdd001f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdd04a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 0x7fdd001656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdd04a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdd000b4500 con 0x7fdd00162c80 2024-04-15T15:20:46.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdd04a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd000b4220 con 0x7fdd00071f50 2024-04-15T15:20:46.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdd04a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdd000b5670 con 0x7fdd000afd00 2024-04-15T15:20:46.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdcff200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 0x7fdd001656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdcff200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 0x7fdd001656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56706/0 (socket says 172.21.15.49:56706) 2024-04-15T15:20:46.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdcff200640 1 -- 172.21.15.49:0/3040246766 learned_addr learned my addr 172.21.15.49:0/3040246766 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:46.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdcfe800640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd00071f50 0x7fdd001ed470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdcfde00640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd000afd00 0x7fdd001f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdcff200640 1 -- 172.21.15.49:0/3040246766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd000afd00 msgr2=0x7fdd001f9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdcff200640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd000afd00 0x7fdd001f9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdcff200640 1 -- 172.21.15.49:0/3040246766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd00071f50 msgr2=0x7fdd001ed470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdcff200640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd00071f50 0x7fdd001ed470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.003+0000 7fdcff200640 1 -- 172.21.15.49:0/3040246766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd00164600 con 0x7fdd00162c80 2024-04-15T15:20:46.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.004+0000 7fdcff200640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 0x7fdd001656a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c7f0 tx=0x7fdcf800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.004+0000 7fdcf6a00640 1 -- 172.21.15.49:0/3040246766 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8018070 con 0x7fdd00162c80 2024-04-15T15:20:46.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.004+0000 7fdcf6a00640 1 -- 172.21.15.49:0/3040246766 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf800a5e0 con 0x7fdd00162c80 2024-04-15T15:20:46.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.004+0000 7fdd04a00640 1 -- 172.21.15.49:0/3040246766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd001648e0 con 0x7fdd00162c80 2024-04-15T15:20:46.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.004+0000 7fdcf6a00640 1 -- 172.21.15.49:0/3040246766 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800a9d0 con 0x7fdd00162c80 2024-04-15T15:20:46.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.004+0000 7fdd04a00640 1 -- 172.21.15.49:0/3040246766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd00164ea0 con 0x7fdd00162c80 2024-04-15T15:20:46.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.006+0000 7fdd04a00640 1 -- 172.21.15.49:0/3040246766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd000755c0 con 0x7fdd00162c80 2024-04-15T15:20:46.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.007+0000 7fdcf6a00640 1 -- 172.21.15.49:0/3040246766 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fdcf801b070 con 0x7fdd00162c80 2024-04-15T15:20:46.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.007+0000 7fdcf6a00640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdce006c980 0x7fdce006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.008+0000 7fdcf6a00640 1 -- 172.21.15.49:0/3040246766 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fdcf808b1a0 con 0x7fdd00162c80 2024-04-15T15:20:46.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.008+0000 7fdcfe800640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdce006c980 0x7fdce006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.008+0000 7fdcfe800640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdce006c980 0x7fdce006ee40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0036000 tx=0x7fdcf0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.012+0000 7fdcf6a00640 1 -- 172.21.15.49:0/3040246766 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd00163060 con 0x7fdd00162c80 2024-04-15T15:20:46.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.256+0000 7fdd04a00640 1 -- 172.21.15.49:0/3040246766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd000cd430 con 0x7fdd00162c80 2024-04-15T15:20:46.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.257+0000 7fdcf6a00640 1 -- 172.21.15.49:0/3040246766 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5188 (secure 0 0 0) 0x7fdcf8059610 con 0x7fdd00162c80 2024-04-15T15:20:46.259 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:46.259 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:00.037472+0000","modified":"2024-04-15T15:20:32.648230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/3168788676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":3168788676},{"type":"v1","addr":"172.21.15.49:6837","nonce":3168788676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5445,"qdb_cluster":[5445]},"id":2}]} 2024-04-15T15:20:46.259 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-04-15T15:20:46.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.259+0000 7fdd04a00640 1 -- 172.21.15.49:0/3040246766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdce006c980 msgr2=0x7fdce006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdce006c980 0x7fdce006ee40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0036000 tx=0x7fdcf0008030 comp rx=0 tx=0).stop 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 -- 172.21.15.49:0/3040246766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 msgr2=0x7fdd001656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 0x7fdd001656a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c7f0 tx=0x7fdcf800ccb0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 -- 172.21.15.49:0/3040246766 shutdown_connections 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdce006c980 0x7fdce006ee40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd000afd00 0x7fdd001f9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd00162c80 0x7fdd001656a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 --2- 172.21.15.49:0/3040246766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd00071f50 0x7fdd001ed470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 -- 172.21.15.49:0/3040246766 >> 172.21.15.49:0/3040246766 conn(0x7fdd0006b930 msgr2=0x7fdd0006dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 -- 172.21.15.49:0/3040246766 shutdown_connections 2024-04-15T15:20:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.260+0000 7fdd04a00640 1 -- 172.21.15.49:0/3040246766 wait complete. 2024-04-15T15:20:46.281 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.501+0000 7fc41be00640 1 Processor -- start 2024-04-15T15:20:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.502+0000 7fc41be00640 1 -- start start 2024-04-15T15:20:46.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.503+0000 7fc41be00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc41c148130 0x7fc41c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.504+0000 7fc41be00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c072660 0x7fc41c0af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.504+0000 7fc41be00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc41c162940 con 0x7fc41c162380 2024-04-15T15:20:46.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.504+0000 7fc41be00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fc41c148510 con 0x7fc41c148130 2024-04-15T15:20:46.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.504+0000 7fc41be00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc41c15fd70 con 0x7fc41c072660 2024-04-15T15:20:46.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.504+0000 7fc41b400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c072660 0x7fc41c0af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.504+0000 7fc41aa00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc41c148130 0x7fc41c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.504+0000 7fc41b400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c072660 0x7fc41c0af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:40722/0 (socket says 172.21.15.49:40722) 2024-04-15T15:20:46.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.504+0000 7fc41aa00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc41c148130 0x7fc41c15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:43874/0 (socket says 172.21.15.49:43874) 2024-04-15T15:20:46.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.504+0000 7fc41b400640 1 -- 172.21.15.49:0/2822328104 learned_addr learned my addr 172.21.15.49:0/2822328104 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:46.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.505+0000 7fc41b400640 1 -- 172.21.15.49:0/2822328104 >> v2:172.21.15.107:3300/0 conn(0x7fc41c148130 msgr2=0x7fc41c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:46.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.505+0000 7fc41b400640 1 --2- 172.21.15.49:0/2822328104 >> v2:172.21.15.107:3300/0 conn(0x7fc41c148130 0x7fc41c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.505+0000 7fc41b400640 1 -- 172.21.15.49:0/2822328104 >> v1:172.21.15.107:6789/0 conn(0x7fc41c162380 legacy=0x7fc41c071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.505+0000 7fc41b400640 1 -- 172.21.15.49:0/2822328104 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41c162c20 con 0x7fc41c072660 2024-04-15T15:20:46.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.505+0000 7fc41aa00640 1 --2- 172.21.15.49:0/2822328104 >> v2:172.21.15.107:3300/0 conn(0x7fc41c148130 0x7fc41c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:46.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.507+0000 7fc41b400640 1 --2- 172.21.15.49:0/2822328104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c072660 0x7fc41c0af1b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc414009e50 tx=0x7fc41402bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90a91f11dbcb9122 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.507+0000 7fc41a000640 1 -- 172.21.15.49:0/2822328104 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41402ca80 con 0x7fc41c072660 2024-04-15T15:20:46.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.507+0000 7fc41a000640 1 -- 172.21.15.49:0/2822328104 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41402cc20 con 0x7fc41c072660 2024-04-15T15:20:46.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.507+0000 7fc41a000640 1 -- 172.21.15.49:0/2822328104 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4140343f0 con 0x7fc41c072660 2024-04-15T15:20:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.508+0000 7fc41be00640 1 -- 172.21.15.49:0/2822328104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c072660 msgr2=0x7fc41c0af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.508+0000 7fc41be00640 1 --2- 172.21.15.49:0/2822328104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c072660 0x7fc41c0af1b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc414009e50 tx=0x7fc41402bee0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.508+0000 7fc41be00640 1 -- 172.21.15.49:0/2822328104 shutdown_connections 2024-04-15T15:20:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.508+0000 7fc41be00640 1 --2- 172.21.15.49:0/2822328104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c072660 0x7fc41c0af1b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.508+0000 7fc41be00640 1 --2- 172.21.15.49:0/2822328104 >> v2:172.21.15.107:3300/0 conn(0x7fc41c148130 0x7fc41c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.508+0000 7fc41be00640 1 -- 172.21.15.49:0/2822328104 >> 172.21.15.49:0/2822328104 conn(0x7fc41c06b930 msgr2=0x7fc41c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.508+0000 7fc41be00640 1 -- 172.21.15.49:0/2822328104 shutdown_connections 2024-04-15T15:20:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.508+0000 7fc41be00640 1 -- 172.21.15.49:0/2822328104 wait complete. 2024-04-15T15:20:46.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.510+0000 7fc41be00640 1 Processor -- start 2024-04-15T15:20:46.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.510+0000 7fc41be00640 1 -- start start 2024-04-15T15:20:46.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc41c072660 0x7fc41c13aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41be00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c148130 0x7fc41c146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41c162380 0x7fc41c147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41be00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c0aeb90 con 0x7fc41c162380 2024-04-15T15:20:46.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c1631e0 con 0x7fc41c072660 2024-04-15T15:20:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41be00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc41c0aed10 con 0x7fc41c148130 2024-04-15T15:20:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41c162380 0x7fc41c147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41c162380 0x7fc41c147340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56708/0 (socket says 172.21.15.49:56708) 2024-04-15T15:20:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41b400640 1 -- 172.21.15.49:0/206590676 learned_addr learned my addr 172.21.15.49:0/206590676 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41aa00640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc41c072660 0x7fc41c13aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc412000640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c148130 0x7fc41c146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41b400640 1 -- 172.21.15.49:0/206590676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c148130 msgr2=0x7fc41c146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41b400640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c148130 0x7fc41c146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41b400640 1 -- 172.21.15.49:0/206590676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc41c072660 msgr2=0x7fc41c13aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41b400640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc41c072660 0x7fc41c13aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.511+0000 7fc41b400640 1 -- 172.21.15.49:0/206590676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41c0cf980 con 0x7fc41c162380 2024-04-15T15:20:46.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.512+0000 7fc41b400640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41c162380 0x7fc41c147340 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc414036040 tx=0x7fc414004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.512+0000 7fc413400640 1 -- 172.21.15.49:0/206590676 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414043020 con 0x7fc41c162380 2024-04-15T15:20:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.512+0000 7fc413400640 1 -- 172.21.15.49:0/206590676 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41403e450 con 0x7fc41c162380 2024-04-15T15:20:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.512+0000 7fc413400640 1 -- 172.21.15.49:0/206590676 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414009510 con 0x7fc41c162380 2024-04-15T15:20:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.512+0000 7fc41be00640 1 -- 172.21.15.49:0/206590676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41c0cfc60 con 0x7fc41c162380 2024-04-15T15:20:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.512+0000 7fc41be00640 1 -- 172.21.15.49:0/206590676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41c13b430 con 0x7fc41c162380 2024-04-15T15:20:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.513+0000 7fc41be00640 1 -- 172.21.15.49:0/206590676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41c0755c0 con 0x7fc41c162380 2024-04-15T15:20:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.514+0000 7fc413400640 1 -- 172.21.15.49:0/206590676 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc414046020 con 0x7fc41c162380 2024-04-15T15:20:46.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.515+0000 7fc413400640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc3f806c980 0x7fc3f806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.515+0000 7fc413400640 1 -- 172.21.15.49:0/206590676 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fc4140ae480 con 0x7fc41c162380 2024-04-15T15:20:46.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.516+0000 7fc41aa00640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc3f806c980 0x7fc3f806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.516+0000 7fc41aa00640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc3f806c980 0x7fc3f806ee40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009560 tx=0x7fc40c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.520+0000 7fc413400640 1 -- 172.21.15.49:0/206590676 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc41404d030 con 0x7fc41c162380 2024-04-15T15:20:46.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.726+0000 7fc41be00640 1 -- 172.21.15.49:0/206590676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc41c06f6d0 con 0x7fc41c162380 2024-04-15T15:20:46.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.728+0000 7fc413400640 1 -- 172.21.15.49:0/206590676 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16798 (secure 0 0 0) 0x7fc41407c7f0 con 0x7fc41c162380 2024-04-15T15:20:46.730 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:46.730 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:28.618835+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6837/1023024717":"2024-04-16T15:20:28.618766+0000","172.21.15.49:6836/1023024717":"2024-04-16T15:20:28.618766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 -- 172.21.15.49:0/206590676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc3f806c980 msgr2=0x7fc3f806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc3f806c980 0x7fc3f806ee40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009560 tx=0x7fc40c008040 comp rx=0 tx=0).stop 2024-04-15T15:20:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 -- 172.21.15.49:0/206590676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41c162380 msgr2=0x7fc41c147340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41c162380 0x7fc41c147340 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc414036040 tx=0x7fc414004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 -- 172.21.15.49:0/206590676 shutdown_connections 2024-04-15T15:20:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc3f806c980 0x7fc3f806ee40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41c148130 0x7fc41c146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc41c162380 0x7fc41c147340 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 --2- 172.21.15.49:0/206590676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc41c072660 0x7fc41c13aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 -- 172.21.15.49:0/206590676 >> 172.21.15.49:0/206590676 conn(0x7fc41c06b930 msgr2=0x7fc41c15e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 -- 172.21.15.49:0/206590676 shutdown_connections 2024-04-15T15:20:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.731+0000 7fc41be00640 1 -- 172.21.15.49:0/206590676 wait complete. 2024-04-15T15:20:46.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:18:38.806030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:19:58.063498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:19:59.009342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:20:46.752 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:20:46.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.974+0000 7f6be8e00640 1 Processor -- start 2024-04-15T15:20:46.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.974+0000 7f6be8e00640 1 -- start start 2024-04-15T15:20:46.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.975+0000 7f6be8e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6be40d18d0 0x7f6be40d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.975+0000 7f6be8e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be415f130 con 0x7f6be40cf1a0 2024-04-15T15:20:46.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.975+0000 7f6be8e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be40d3210 con 0x7f6be415ed50 2024-04-15T15:20:46.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.975+0000 7f6be8e00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f6be40b7e40 con 0x7f6be40d18d0 2024-04-15T15:20:46.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.976+0000 7f6be1400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6be40d18d0 0x7f6be40d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.976+0000 7f6be2800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6be40cf1a0 0x7f6be40cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56562/0 (socket says 172.21.15.49:56562) 2024-04-15T15:20:46.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.976+0000 7f6be2800640 1 -- 172.21.15.49:0/383926745 learned_addr learned my addr 172.21.15.49:0/383926745 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:46.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.976+0000 7f6be1400640 1 -- 172.21.15.49:0/383926745 >> v1:172.21.15.107:6789/0 conn(0x7f6be415ed50 legacy=0x7f6be40d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:46.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.976+0000 7f6be1400640 1 -- 172.21.15.49:0/383926745 >> v1:172.21.15.49:6789/0 conn(0x7f6be40cf1a0 legacy=0x7f6be40cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.976+0000 7f6be1400640 1 -- 172.21.15.49:0/383926745 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be40d34e0 con 0x7f6be40d18d0 2024-04-15T15:20:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.978+0000 7f6be1400640 1 --2- 172.21.15.49:0/383926745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 0x7f6be40d1d50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0097a0 tx=0x7f6bcc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b1932fd1a785a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.978+0000 7f6be0a00640 1 -- 172.21.15.49:0/383926745 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0315a0 con 0x7f6be40d18d0 2024-04-15T15:20:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.978+0000 7f6be0a00640 1 -- 172.21.15.49:0/383926745 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bcc037070 con 0x7f6be40d18d0 2024-04-15T15:20:46.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.978+0000 7f6be0a00640 1 -- 172.21.15.49:0/383926745 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc031960 con 0x7f6be40d18d0 2024-04-15T15:20:46.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.978+0000 7f6be8e00640 1 -- 172.21.15.49:0/383926745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 msgr2=0x7f6be40d1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.978+0000 7f6be8e00640 1 --2- 172.21.15.49:0/383926745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 0x7f6be40d1d50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0097a0 tx=0x7f6bcc02b840 comp rx=0 tx=0).stop 2024-04-15T15:20:46.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.979+0000 7f6be8e00640 1 -- 172.21.15.49:0/383926745 shutdown_connections 2024-04-15T15:20:46.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.979+0000 7f6be8e00640 1 --2- 172.21.15.49:0/383926745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 0x7f6be40d1d50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.979+0000 7f6be8e00640 1 -- 172.21.15.49:0/383926745 >> 172.21.15.49:0/383926745 conn(0x7f6be406d400 msgr2=0x7f6be406d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:46.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.979+0000 7f6be8e00640 1 -- 172.21.15.49:0/383926745 shutdown_connections 2024-04-15T15:20:46.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.979+0000 7f6be8e00640 1 -- 172.21.15.49:0/383926745 wait complete. 2024-04-15T15:20:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.980+0000 7f6be8e00640 1 Processor -- start 2024-04-15T15:20:46.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.981+0000 7f6be8e00640 1 -- start start 2024-04-15T15:20:46.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.981+0000 7f6be8e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6be40cf1a0 0x7f6be40c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.981+0000 7f6be8e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 0x7f6be41e8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.981+0000 7f6be8e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be415ed50 0x7f6be414e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be8e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6be40b79d0 con 0x7f6be40d18d0 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be8e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6be40d3aa0 con 0x7f6be415ed50 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be8e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6be40b7b50 con 0x7f6be40cf1a0 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 0x7f6be41e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 0x7f6be41e8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56714/0 (socket says 172.21.15.49:56714) 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be1400640 1 -- 172.21.15.49:0/117182743 learned_addr learned my addr 172.21.15.49:0/117182743 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be2800640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be415ed50 0x7f6be414e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be1e00640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6be40cf1a0 0x7f6be40c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be1400640 1 -- 172.21.15.49:0/117182743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6be40cf1a0 msgr2=0x7f6be40c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be1400640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6be40cf1a0 0x7f6be40c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be1400640 1 -- 172.21.15.49:0/117182743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be415ed50 msgr2=0x7f6be414e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:46.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be1400640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be415ed50 0x7f6be414e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:46.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be1400640 1 -- 172.21.15.49:0/117182743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be414e330 con 0x7f6be40d18d0 2024-04-15T15:20:46.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.982+0000 7f6be1e00640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6be40cf1a0 0x7f6be40c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:46.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.983+0000 7f6be1400640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 0x7f6be41e8120 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc034040 tx=0x7f6bcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.983+0000 7f6bda000640 1 -- 172.21.15.49:0/117182743 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc032840 con 0x7f6be40d18d0 2024-04-15T15:20:46.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.983+0000 7f6bda000640 1 -- 172.21.15.49:0/117182743 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bcc032a00 con 0x7f6be40d18d0 2024-04-15T15:20:46.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.983+0000 7f6bda000640 1 -- 172.21.15.49:0/117182743 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc032df0 con 0x7f6be40d18d0 2024-04-15T15:20:46.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.983+0000 7f6be8e00640 1 -- 172.21.15.49:0/117182743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be41e8660 con 0x7f6be40d18d0 2024-04-15T15:20:46.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.983+0000 7f6be8e00640 1 -- 172.21.15.49:0/117182743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be40c9eb0 con 0x7f6be40d18d0 2024-04-15T15:20:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.984+0000 7f6be8e00640 1 -- 172.21.15.49:0/117182743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be4068c80 con 0x7f6be40d18d0 2024-04-15T15:20:46.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.985+0000 7f6bda000640 1 -- 172.21.15.49:0/117182743 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6bcc03b3f0 con 0x7f6be40d18d0 2024-04-15T15:20:46.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.986+0000 7f6bda000640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6bc006c980 0x7f6bc006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:46.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.986+0000 7f6bda000640 1 -- 172.21.15.49:0/117182743 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f6bcc02f030 con 0x7f6be40d18d0 2024-04-15T15:20:46.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.986+0000 7f6be1e00640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6bc006c980 0x7f6bc006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:46.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.987+0000 7f6be1e00640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6bc006c980 0x7f6bc006ee40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400a830 tx=0x7f6bd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:46.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:46.990+0000 7f6bda000640 1 -- 172.21.15.49:0/117182743 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6bcc040070 con 0x7f6be40d18d0 2024-04-15T15:20:47.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.192+0000 7f6be8e00640 1 -- 172.21.15.49:0/117182743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6be40b9f10 con 0x7f6be40d18d0 2024-04-15T15:20:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.193+0000 7f6bda000640 1 -- 172.21.15.49:0/117182743 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f6bcc07bf90 con 0x7f6be40d18d0 2024-04-15T15:20:47.195 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-04-15T15:20:47.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.195+0000 7f6be8e00640 1 -- 172.21.15.49:0/117182743 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6bc006c980 msgr2=0x7f6bc006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.195+0000 7f6be8e00640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6bc006c980 0x7f6bc006ee40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400a830 tx=0x7f6bd4008040 comp rx=0 tx=0).stop 2024-04-15T15:20:47.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.195+0000 7f6be8e00640 1 -- 172.21.15.49:0/117182743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 msgr2=0x7f6be41e8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.195+0000 7f6be8e00640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 0x7f6be41e8120 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc034040 tx=0x7f6bcc0046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.196+0000 7f6be8e00640 1 -- 172.21.15.49:0/117182743 shutdown_connections 2024-04-15T15:20:47.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.196+0000 7f6be8e00640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6bc006c980 0x7f6bc006ee40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.196+0000 7f6be8e00640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be415ed50 0x7f6be414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.196+0000 7f6be8e00640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be40d18d0 0x7f6be41e8120 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.196+0000 7f6be8e00640 1 --2- 172.21.15.49:0/117182743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6be40cf1a0 0x7f6be40c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.196+0000 7f6be8e00640 1 -- 172.21.15.49:0/117182743 >> 172.21.15.49:0/117182743 conn(0x7f6be406d400 msgr2=0x7f6be406d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:47.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.196+0000 7f6be8e00640 1 -- 172.21.15.49:0/117182743 shutdown_connections 2024-04-15T15:20:47.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.196+0000 7f6be8e00640 1 -- 172.21.15.49:0/117182743 wait complete. 2024-04-15T15:20:47.216 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:20:47.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.451+0000 7f81dda00640 1 Processor -- start 2024-04-15T15:20:47.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.451+0000 7f81dda00640 1 -- start start 2024-04-15T15:20:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.452+0000 7f81dda00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f81d815c680 0x7f81d8162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.453+0000 7f81dda00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81d80b6c70 0x7f81d815f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.453+0000 7f81dda00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f81d815fd90 0x7f81d80d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.453+0000 7f81dda00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f81d80b70d0 con 0x7f81d815c680 2024-04-15T15:20:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.453+0000 7f81dda00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f81d8162dd0 con 0x7f81d815fd90 2024-04-15T15:20:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.453+0000 7f81dda00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f81d81630b0 con 0x7f81d80b6c70 2024-04-15T15:20:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.453+0000 7f81d7e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f81d815c680 0x7f81d8162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.453+0000 7f81d7e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f81d815c680 0x7f81d8162730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56728/0 (socket says 172.21.15.49:56728) 2024-04-15T15:20:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.453+0000 7f81d7e00640 1 -- 172.21.15.49:0/1411000071 learned_addr learned my addr 172.21.15.49:0/1411000071 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.453+0000 7f81d7400640 1 --2- 172.21.15.49:0/1411000071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81d80b6c70 0x7f81d815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.453+0000 7f81dd000640 1 --2- 172.21.15.49:0/1411000071 >> v2:172.21.15.107:3300/0 conn(0x7f81d815fd90 0x7f81d80d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.454+0000 7f81d7e00640 1 -- 172.21.15.49:0/1411000071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81d80b6c70 msgr2=0x7f81d815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:47.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.454+0000 7f81d7e00640 1 --2- 172.21.15.49:0/1411000071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81d80b6c70 0x7f81d815f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.454+0000 7f81d7e00640 1 -- 172.21.15.49:0/1411000071 >> v2:172.21.15.107:3300/0 conn(0x7f81d815fd90 msgr2=0x7f81d80d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:47.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.454+0000 7f81d7e00640 1 --2- 172.21.15.49:0/1411000071 >> v2:172.21.15.107:3300/0 conn(0x7f81d815fd90 0x7f81d80d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.454+0000 7f81d7e00640 1 -- 172.21.15.49:0/1411000071 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d8163390 con 0x7f81d815c680 2024-04-15T15:20:47.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.455+0000 7f81d7e00640 1 --2- 172.21.15.49:0/1411000071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815c680 0x7f81d8162730 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f81d80ba1f0 tx=0x7f81c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa11240d0e6ae566 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.455+0000 7f81d6a00640 1 -- 172.21.15.49:0/1411000071 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8035020 con 0x7f81d815c680 2024-04-15T15:20:47.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.455+0000 7f81d6a00640 1 -- 172.21.15.49:0/1411000071 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c8031870 con 0x7f81d815c680 2024-04-15T15:20:47.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.455+0000 7f81d6a00640 1 -- 172.21.15.49:0/1411000071 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8031c40 con 0x7f81d815c680 2024-04-15T15:20:47.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.456+0000 7f81dda00640 1 -- 172.21.15.49:0/1411000071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815c680 msgr2=0x7f81d8162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.456+0000 7f81dda00640 1 --2- 172.21.15.49:0/1411000071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815c680 0x7f81d8162730 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f81d80ba1f0 tx=0x7f81c802b910 comp rx=0 tx=0).stop 2024-04-15T15:20:47.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.456+0000 7f81dda00640 1 -- 172.21.15.49:0/1411000071 shutdown_connections 2024-04-15T15:20:47.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.456+0000 7f81dda00640 1 --2- 172.21.15.49:0/1411000071 >> v2:172.21.15.107:3300/0 conn(0x7f81d815fd90 0x7f81d80d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.456+0000 7f81dda00640 1 --2- 172.21.15.49:0/1411000071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81d80b6c70 0x7f81d815f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.456+0000 7f81dda00640 1 --2- 172.21.15.49:0/1411000071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815c680 0x7f81d8162730 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.456+0000 7f81dda00640 1 -- 172.21.15.49:0/1411000071 >> 172.21.15.49:0/1411000071 conn(0x7f81d8073480 msgr2=0x7f81d8073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:47.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.456+0000 7f81dda00640 1 -- 172.21.15.49:0/1411000071 shutdown_connections 2024-04-15T15:20:47.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.456+0000 7f81dda00640 1 -- 172.21.15.49:0/1411000071 wait complete. 2024-04-15T15:20:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.458+0000 7f81dda00640 1 Processor -- start 2024-04-15T15:20:47.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.458+0000 7f81dda00640 1 -- start start 2024-04-15T15:20:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.459+0000 7f81dda00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81d80b6c70 0x7f81d81f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.459+0000 7f81dda00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81d815c680 0x7f81d815afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.459+0000 7f81dda00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815fd90 0x7f81d81555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.459+0000 7f81dda00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f81d80d3090 con 0x7f81d815fd90 2024-04-15T15:20:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.459+0000 7f81dda00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f81d815ca60 con 0x7f81d815c680 2024-04-15T15:20:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.459+0000 7f81dda00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f81d80d3210 con 0x7f81d80b6c70 2024-04-15T15:20:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.459+0000 7f81dd000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815fd90 0x7f81d81555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.459+0000 7f81dd000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815fd90 0x7f81d81555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56730/0 (socket says 172.21.15.49:56730) 2024-04-15T15:20:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.459+0000 7f81dd000640 1 -- 172.21.15.49:0/997982608 learned_addr learned my addr 172.21.15.49:0/997982608 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:47.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.460+0000 7f81d7400640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81d815c680 0x7f81d815afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.460+0000 7f81d7e00640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81d80b6c70 0x7f81d81f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.460+0000 7f81dd000640 1 -- 172.21.15.49:0/997982608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81d80b6c70 msgr2=0x7f81d81f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.460+0000 7f81dd000640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81d80b6c70 0x7f81d81f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.460+0000 7f81dd000640 1 -- 172.21.15.49:0/997982608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81d815c680 msgr2=0x7f81d815afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.460+0000 7f81dd000640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81d815c680 0x7f81d815afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.460+0000 7f81dd000640 1 -- 172.21.15.49:0/997982608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d80c83b0 con 0x7f81d815fd90 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.460+0000 7f81dd000640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815fd90 0x7f81d81555d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f81d000d5a0 tx=0x7f81d000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.461+0000 7f81cfe00640 1 -- 172.21.15.49:0/997982608 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d00125f0 con 0x7f81d815fd90 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.461+0000 7f81cfe00640 1 -- 172.21.15.49:0/997982608 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d0019070 con 0x7f81d815fd90 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.461+0000 7f81cfe00640 1 -- 172.21.15.49:0/997982608 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d00129b0 con 0x7f81d815fd90 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.461+0000 7f81dda00640 1 -- 172.21.15.49:0/997982608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d81f84d0 con 0x7f81d815fd90 2024-04-15T15:20:47.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.461+0000 7f81dda00640 1 -- 172.21.15.49:0/997982608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d80c8620 con 0x7f81d815fd90 2024-04-15T15:20:47.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.462+0000 7f81dda00640 1 -- 172.21.15.49:0/997982608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d80c8820 con 0x7f81d815fd90 2024-04-15T15:20:47.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.463+0000 7f81cfe00640 1 -- 172.21.15.49:0/997982608 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f81d0012b50 con 0x7f81d815fd90 2024-04-15T15:20:47.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.464+0000 7f81cfe00640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81b806c980 0x7f81b806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.464+0000 7f81cfe00640 1 -- 172.21.15.49:0/997982608 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f81d008bff0 con 0x7f81d815fd90 2024-04-15T15:20:47.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.464+0000 7f81d7e00640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81b806c980 0x7f81b806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.465+0000 7f81d7e00640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81b806c980 0x7f81b806ee40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f81c8036000 tx=0x7f81c803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.468+0000 7f81cfe00640 1 -- 172.21.15.49:0/997982608 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81d000f300 con 0x7f81d815fd90 2024-04-15T15:20:47.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.670+0000 7f81dda00640 1 -- 172.21.15.49:0/997982608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f81d806dbe0 con 0x7f81d815fd90 2024-04-15T15:20:47.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.671+0000 7f81cfe00640 1 -- 172.21.15.49:0/997982608 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7f81d005a340 con 0x7f81d815fd90 2024-04-15T15:20:47.673 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-04-15T15:20:47.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.673+0000 7f81dda00640 1 -- 172.21.15.49:0/997982608 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81b806c980 msgr2=0x7f81b806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.673+0000 7f81dda00640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81b806c980 0x7f81b806ee40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f81c8036000 tx=0x7f81c803c040 comp rx=0 tx=0).stop 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.674+0000 7f81dda00640 1 -- 172.21.15.49:0/997982608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815fd90 msgr2=0x7f81d81555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.674+0000 7f81dda00640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815fd90 0x7f81d81555d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f81d000d5a0 tx=0x7f81d000da60 comp rx=0 tx=0).stop 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.674+0000 7f81dda00640 1 -- 172.21.15.49:0/997982608 shutdown_connections 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.674+0000 7f81dda00640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f81b806c980 0x7f81b806ee40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.674+0000 7f81dda00640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81d815fd90 0x7f81d81555d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.674+0000 7f81dda00640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81d815c680 0x7f81d815afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.674+0000 7f81dda00640 1 --2- 172.21.15.49:0/997982608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81d80b6c70 0x7f81d81f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.674+0000 7f81dda00640 1 -- 172.21.15.49:0/997982608 >> 172.21.15.49:0/997982608 conn(0x7f81d8073480 msgr2=0x7f81d80cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.674+0000 7f81dda00640 1 -- 172.21.15.49:0/997982608 shutdown_connections 2024-04-15T15:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.674+0000 7f81dda00640 1 -- 172.21.15.49:0/997982608 wait complete. 2024-04-15T15:20:47.695 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:20:47.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.911+0000 7f4790a00640 1 Processor -- start 2024-04-15T15:20:47.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.911+0000 7f4790a00640 1 -- start start 2024-04-15T15:20:47.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.913+0000 7f4790a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f478c1478b0 0x7f478c0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.913+0000 7f4790a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f478c0aebd0 con 0x7f478c147c90 2024-04-15T15:20:47.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.913+0000 7f4790a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f478c0b0130 con 0x7f478c06f920 2024-04-15T15:20:47.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.913+0000 7f4790a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f478c0aee50 con 0x7f478c1478b0 2024-04-15T15:20:47.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.913+0000 7f478b400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f478c1478b0 0x7f478c0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.914+0000 7f4783e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f478c147c90 0x7f478c06f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56578/0 (socket says 172.21.15.49:56578) 2024-04-15T15:20:47.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.914+0000 7f4783e00640 1 -- 172.21.15.49:0/445059965 learned_addr learned my addr 172.21.15.49:0/445059965 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:47.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.914+0000 7f478b400640 1 -- 172.21.15.49:0/445059965 >> v1:172.21.15.107:6789/0 conn(0x7f478c06f920 legacy=0x7f478c1626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:47.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.914+0000 7f478aa00640 1 -- 172.21.15.49:0/445059965 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 780588886 0 0) 0x7f4774001110 con 0x7f478c147c90 2024-04-15T15:20:47.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.914+0000 7f478b400640 1 -- 172.21.15.49:0/445059965 >> v1:172.21.15.49:6789/0 conn(0x7f478c147c90 legacy=0x7f478c06f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.914+0000 7f478b400640 1 -- 172.21.15.49:0/445059965 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c0af110 con 0x7f478c1478b0 2024-04-15T15:20:47.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.916+0000 7f478b400640 1 --2- 172.21.15.49:0/445059965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c1478b0 0x7f478c0afbf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f477c0097a0 tx=0x7f477c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6e30d9e6dd0d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.916+0000 7f478aa00640 1 -- 172.21.15.49:0/445059965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c034070 con 0x7f478c1478b0 2024-04-15T15:20:47.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.916+0000 7f478aa00640 1 -- 172.21.15.49:0/445059965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477c030610 con 0x7f478c1478b0 2024-04-15T15:20:47.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.916+0000 7f478aa00640 1 -- 172.21.15.49:0/445059965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c0309e0 con 0x7f478c1478b0 2024-04-15T15:20:47.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.917+0000 7f4790a00640 1 -- 172.21.15.49:0/445059965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c1478b0 msgr2=0x7f478c0afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.917+0000 7f4790a00640 1 --2- 172.21.15.49:0/445059965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c1478b0 0x7f478c0afbf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f477c0097a0 tx=0x7f477c02b960 comp rx=0 tx=0).stop 2024-04-15T15:20:47.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.917+0000 7f4790a00640 1 -- 172.21.15.49:0/445059965 shutdown_connections 2024-04-15T15:20:47.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.917+0000 7f4790a00640 1 --2- 172.21.15.49:0/445059965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c1478b0 0x7f478c0afbf0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.917+0000 7f4790a00640 1 -- 172.21.15.49:0/445059965 >> 172.21.15.49:0/445059965 conn(0x7f478c06b930 msgr2=0x7f478c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:47.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.917+0000 7f4790a00640 1 -- 172.21.15.49:0/445059965 shutdown_connections 2024-04-15T15:20:47.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.917+0000 7f4790a00640 1 -- 172.21.15.49:0/445059965 wait complete. 2024-04-15T15:20:47.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.919+0000 7f4790a00640 1 Processor -- start 2024-04-15T15:20:47.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.919+0000 7f4790a00640 1 -- start start 2024-04-15T15:20:47.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.920+0000 7f4790a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f478c06f920 0x7f478c13f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.920+0000 7f4790a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f478c1478b0 0x7f478c146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.920+0000 7f4790a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c147c90 0x7f478c146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.920+0000 7f4790a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f478c162dd0 con 0x7f478c147c90 2024-04-15T15:20:47.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.920+0000 7f4790a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f478c0af6d0 con 0x7f478c1478b0 2024-04-15T15:20:47.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.920+0000 7f4790a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f478c162f50 con 0x7f478c06f920 2024-04-15T15:20:47.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.920+0000 7f478be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c147c90 0x7f478c146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.920+0000 7f478be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c147c90 0x7f478c146850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56750/0 (socket says 172.21.15.49:56750) 2024-04-15T15:20:47.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.920+0000 7f478be00640 1 -- 172.21.15.49:0/2111232511 learned_addr learned my addr 172.21.15.49:0/2111232511 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.921+0000 7f4783e00640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f478c1478b0 0x7f478c146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.921+0000 7f478be00640 1 -- 172.21.15.49:0/2111232511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f478c06f920 msgr2=0x7f478c13f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.921+0000 7f478be00640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f478c06f920 0x7f478c13f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.921+0000 7f478be00640 1 -- 172.21.15.49:0/2111232511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f478c1478b0 msgr2=0x7f478c146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.921+0000 7f478be00640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f478c1478b0 0x7f478c146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.921+0000 7f478be00640 1 -- 172.21.15.49:0/2111232511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c146eb0 con 0x7f478c147c90 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.921+0000 7f478be00640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c147c90 0x7f478c146850 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f478400c7a0 tx=0x7f478400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.921+0000 7f4783400640 1 -- 172.21.15.49:0/2111232511 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784003b70 con 0x7f478c147c90 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.922+0000 7f4783400640 1 -- 172.21.15.49:0/2111232511 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4784003d30 con 0x7f478c147c90 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.922+0000 7f4783400640 1 -- 172.21.15.49:0/2111232511 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478400aa90 con 0x7f478c147c90 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.922+0000 7f4790a00640 1 -- 172.21.15.49:0/2111232511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478c13cb60 con 0x7f478c147c90 2024-04-15T15:20:47.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.922+0000 7f4790a00640 1 -- 172.21.15.49:0/2111232511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f478c13d120 con 0x7f478c147c90 2024-04-15T15:20:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.923+0000 7f4790a00640 1 -- 172.21.15.49:0/2111232511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f478c13d320 con 0x7f478c147c90 2024-04-15T15:20:47.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.924+0000 7f4783400640 1 -- 172.21.15.49:0/2111232511 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f478400ac30 con 0x7f478c147c90 2024-04-15T15:20:47.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.925+0000 7f4783400640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f476806c980 0x7f476806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:47.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.925+0000 7f4783400640 1 -- 172.21.15.49:0/2111232511 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f478408bbe0 con 0x7f478c147c90 2024-04-15T15:20:47.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.925+0000 7f478b400640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f476806c980 0x7f476806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:47.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.926+0000 7f478b400640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f476806c980 0x7f476806ee40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f477c036000 tx=0x7f477c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:47.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:47.929+0000 7f4783400640 1 -- 172.21.15.49:0/2111232511 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f478c13d320 con 0x7f478c147c90 2024-04-15T15:20:48.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.134+0000 7f4790a00640 1 -- 172.21.15.49:0/2111232511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f478c0755c0 con 0x7f478c147c90 2024-04-15T15:20:48.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.135+0000 7f4783400640 1 -- 172.21.15.49:0/2111232511 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f478c0b0130 con 0x7f478c147c90 2024-04-15T15:20:48.137 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 -- 172.21.15.49:0/2111232511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f476806c980 msgr2=0x7f476806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f476806c980 0x7f476806ee40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f477c036000 tx=0x7f477c030000 comp rx=0 tx=0).stop 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 -- 172.21.15.49:0/2111232511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c147c90 msgr2=0x7f478c146850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c147c90 0x7f478c146850 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f478400c7a0 tx=0x7f478400cc60 comp rx=0 tx=0).stop 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 -- 172.21.15.49:0/2111232511 shutdown_connections 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f476806c980 0x7f476806ee40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f478c147c90 0x7f478c146850 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f478c1478b0 0x7f478c146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 --2- 172.21.15.49:0/2111232511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f478c06f920 0x7f478c13f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 -- 172.21.15.49:0/2111232511 >> 172.21.15.49:0/2111232511 conn(0x7f478c06b930 msgr2=0x7f478c0b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 -- 172.21.15.49:0/2111232511 shutdown_connections 2024-04-15T15:20:48.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.138+0000 7f4790a00640 1 -- 172.21.15.49:0/2111232511 wait complete. 2024-04-15T15:20:48.159 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:48.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.376+0000 7f54f5800640 1 Processor -- start 2024-04-15T15:20:48.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.377+0000 7f54f5800640 1 -- start start 2024-04-15T15:20:48.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.378+0000 7f54f5800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f0162020 0x7f54f0162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.378+0000 7f54f5800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f54f00cf110 0x7f54f00d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.378+0000 7f54f5800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f00d0090 con 0x7f54f00d1ac0 2024-04-15T15:20:48.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.378+0000 7f54f5800640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f54f00d1f50 con 0x7f54f00cf110 2024-04-15T15:20:48.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.378+0000 7f54f5800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f54f0163440 con 0x7f54f0162020 2024-04-15T15:20:48.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.379+0000 7f54ef400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f54f00cf110 0x7f54f00d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.379+0000 7f54efe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f0162020 0x7f54f0162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.379+0000 7f54ef400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f54f00cf110 0x7f54f00d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:43920/0 (socket says 172.21.15.49:43920) 2024-04-15T15:20:48.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.379+0000 7f54ef400640 1 -- 172.21.15.49:0/1715654016 learned_addr learned my addr 172.21.15.49:0/1715654016 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:48.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.380+0000 7f54efe00640 1 -- 172.21.15.49:0/1715654016 >> v2:172.21.15.107:3300/0 conn(0x7f54f00cf110 msgr2=0x7f54f00d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.380+0000 7f54efe00640 1 --2- 172.21.15.49:0/1715654016 >> v2:172.21.15.107:3300/0 conn(0x7f54f00cf110 0x7f54f00d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.380+0000 7f54efe00640 1 -- 172.21.15.49:0/1715654016 >> v1:172.21.15.107:6789/0 conn(0x7f54f00d1ac0 legacy=0x7f54f00b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.380+0000 7f54efe00640 1 -- 172.21.15.49:0/1715654016 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f00cfad0 con 0x7f54f0162020 2024-04-15T15:20:48.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.380+0000 7f54ef400640 1 --2- 172.21.15.49:0/1715654016 >> v2:172.21.15.107:3300/0 conn(0x7f54f00cf110 0x7f54f00d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:20:48.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.381+0000 7f54efe00640 1 --2- 172.21.15.49:0/1715654016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f0162020 0x7f54f0162f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009d90 tx=0x7f54e002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2740344535e6c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.382+0000 7f54eea00640 1 -- 172.21.15.49:0/1715654016 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0039070 con 0x7f54f0162020 2024-04-15T15:20:48.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.382+0000 7f54eea00640 1 -- 172.21.15.49:0/1715654016 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e002cc50 con 0x7f54f0162020 2024-04-15T15:20:48.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.382+0000 7f54eea00640 1 -- 172.21.15.49:0/1715654016 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0034440 con 0x7f54f0162020 2024-04-15T15:20:48.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.382+0000 7f54f5800640 1 -- 172.21.15.49:0/1715654016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f0162020 msgr2=0x7f54f0162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.382+0000 7f54f5800640 1 --2- 172.21.15.49:0/1715654016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f0162020 0x7f54f0162f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009d90 tx=0x7f54e002bea0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.382+0000 7f54f5800640 1 -- 172.21.15.49:0/1715654016 shutdown_connections 2024-04-15T15:20:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.382+0000 7f54f5800640 1 --2- 172.21.15.49:0/1715654016 >> v2:172.21.15.107:3300/0 conn(0x7f54f00cf110 0x7f54f00d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.382+0000 7f54f5800640 1 --2- 172.21.15.49:0/1715654016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f0162020 0x7f54f0162f00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.382+0000 7f54f5800640 1 -- 172.21.15.49:0/1715654016 >> 172.21.15.49:0/1715654016 conn(0x7f54f006cf00 msgr2=0x7f54f006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.383+0000 7f54f5800640 1 -- 172.21.15.49:0/1715654016 shutdown_connections 2024-04-15T15:20:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.383+0000 7f54f5800640 1 -- 172.21.15.49:0/1715654016 wait complete. 2024-04-15T15:20:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.384+0000 7f54f5800640 1 Processor -- start 2024-04-15T15:20:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.384+0000 7f54f5800640 1 -- start start 2024-04-15T15:20:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.385+0000 7f54f5800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54f00cf110 0x7f54f015ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.385+0000 7f54f5800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f00d1ac0 0x7f54f0160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.385+0000 7f54f5800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f0162020 0x7f54f0154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.385+0000 7f54f5800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f54f00cf880 con 0x7f54f00cf110 2024-04-15T15:20:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.385+0000 7f54f5800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54f00b7b30 con 0x7f54f0162020 2024-04-15T15:20:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.385+0000 7f54f5800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f54f00d2b90 con 0x7f54f00d1ac0 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54efe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54f00cf110 0x7f54f015ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54efe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54f00cf110 0x7f54f015ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56764/0 (socket says 172.21.15.49:56764) 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54efe00640 1 -- 172.21.15.49:0/575668922 learned_addr learned my addr 172.21.15.49:0/575668922 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54f4e00640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f0162020 0x7f54f0154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54efe00640 1 -- 172.21.15.49:0/575668922 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f00d1ac0 msgr2=0x7f54f0160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54ef400640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f00d1ac0 0x7f54f0160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54efe00640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f00d1ac0 0x7f54f0160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54efe00640 1 -- 172.21.15.49:0/575668922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f0162020 msgr2=0x7f54f0154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54efe00640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f0162020 0x7f54f0154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54efe00640 1 -- 172.21.15.49:0/575668922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f01e8370 con 0x7f54f00cf110 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.386+0000 7f54f4e00640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f0162020 0x7f54f0154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.387+0000 7f54efe00640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54f00cf110 0x7f54f015ee80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f54e0037040 tx=0x7f54e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.387+0000 7f54e7e00640 1 -- 172.21.15.49:0/575668922 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0039050 con 0x7f54f00cf110 2024-04-15T15:20:48.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.387+0000 7f54e7e00640 1 -- 172.21.15.49:0/575668922 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e0045070 con 0x7f54f00cf110 2024-04-15T15:20:48.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.387+0000 7f54e7e00640 1 -- 172.21.15.49:0/575668922 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e003e610 con 0x7f54f00cf110 2024-04-15T15:20:48.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.387+0000 7f54f5800640 1 -- 172.21.15.49:0/575668922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f015f3c0 con 0x7f54f00cf110 2024-04-15T15:20:48.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.387+0000 7f54f5800640 1 -- 172.21.15.49:0/575668922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01e88f0 con 0x7f54f00cf110 2024-04-15T15:20:48.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.388+0000 7f54f5800640 1 -- 172.21.15.49:0/575668922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f00755c0 con 0x7f54f00cf110 2024-04-15T15:20:48.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.389+0000 7f54e7e00640 1 -- 172.21.15.49:0/575668922 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f54e003e7b0 con 0x7f54f00cf110 2024-04-15T15:20:48.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.390+0000 7f54e7e00640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54cc06c9d0 0x7f54cc06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.390+0000 7f54e7e00640 1 -- 172.21.15.49:0/575668922 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f54e00ae760 con 0x7f54f00cf110 2024-04-15T15:20:48.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.391+0000 7f54ef400640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54cc06c9d0 0x7f54cc06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.391+0000 7f54ef400640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54cc06c9d0 0x7f54cc06ee90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54f015fde0 tx=0x7f54d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.395+0000 7f54e7e00640 1 -- 172.21.15.49:0/575668922 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54e007cb50 con 0x7f54f00cf110 2024-04-15T15:20:48.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.645+0000 7f54f5800640 1 -- 172.21.15.49:0/575668922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54f00b7650 con 0x7f54f00cf110 2024-04-15T15:20:48.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.646+0000 7f54e7e00640 1 -- 172.21.15.49:0/575668922 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5188 (secure 0 0 0) 0x7f54e007c450 con 0x7f54f00cf110 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:00.037472+0000","modified":"2024-04-15T15:20:32.648230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/3168788676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":3168788676},{"type":"v1","addr":"172.21.15.49:6837","nonce":3168788676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5445,"qdb_cluster":[5445]},"id":2}]} 2024-04-15T15:20:48.648 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 -- 172.21.15.49:0/575668922 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54cc06c9d0 msgr2=0x7f54cc06ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54cc06c9d0 0x7f54cc06ee90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54f015fde0 tx=0x7f54d8008040 comp rx=0 tx=0).stop 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 -- 172.21.15.49:0/575668922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54f00cf110 msgr2=0x7f54f015ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54f00cf110 0x7f54f015ee80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f54e0037040 tx=0x7f54e0004a40 comp rx=0 tx=0).stop 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 -- 172.21.15.49:0/575668922 shutdown_connections 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f54cc06c9d0 0x7f54cc06ee90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54f00d1ac0 0x7f54f0160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f0162020 0x7f54f0154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 --2- 172.21.15.49:0/575668922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54f00cf110 0x7f54f015ee80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 -- 172.21.15.49:0/575668922 >> 172.21.15.49:0/575668922 conn(0x7f54f006cf00 msgr2=0x7f54f00cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 -- 172.21.15.49:0/575668922 shutdown_connections 2024-04-15T15:20:48.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.649+0000 7f54f5800640 1 -- 172.21.15.49:0/575668922 wait complete. 2024-04-15T15:20:48.669 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:48.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.881+0000 7f6b33200640 1 Processor -- start 2024-04-15T15:20:48.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.881+0000 7f6b33200640 1 -- start start 2024-04-15T15:20:48.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.882+0000 7f6b33200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6b340d1c90 0x7f6b34162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.882+0000 7f6b33200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b340cf110 0x7f6b340d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.882+0000 7f6b33200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b340d0090 con 0x7f6b34162020 2024-04-15T15:20:48.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.882+0000 7f6b33200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f6b34163440 con 0x7f6b340d1c90 2024-04-15T15:20:48.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.882+0000 7f6b33200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6b340b7b30 con 0x7f6b340cf110 2024-04-15T15:20:48.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.883+0000 7f6b31e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6b34162020 0x7f6b340d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56588/0 (socket says 172.21.15.49:56588) 2024-04-15T15:20:48.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.883+0000 7f6b31e00640 1 -- 172.21.15.49:0/2064351074 learned_addr learned my addr 172.21.15.49:0/2064351074 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:48.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.883+0000 7f6b31400640 1 --2- 172.21.15.49:0/2064351074 >> v2:172.21.15.107:3300/0 conn(0x7f6b340d1c90 0x7f6b34162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.883+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2720518871 0 0) 0x7f6b24001110 con 0x7f6b34162020 2024-04-15T15:20:48.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.883+0000 7f6b32800640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b340cf110 0x7f6b340d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.883+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b18003520 con 0x7f6b34162020 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.883+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2505024736 0 0) 0x7f6b18003520 con 0x7f6b34162020 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b340cf110 msgr2=0x7f6b340d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b340cf110 0x7f6b340d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 >> v2:172.21.15.107:3300/0 conn(0x7f6b340d1c90 msgr2=0x7f6b34162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 --2- 172.21.15.49:0/2064351074 >> v2:172.21.15.107:3300/0 conn(0x7f6b340d1c90 0x7f6b34162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b340cfad0 con 0x7f6b34162020 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f6b24001840 con 0x7f6b34162020 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 >> v1:172.21.15.49:6789/0 conn(0x7f6b34162020 legacy=0x7f6b340d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b180039b0 0x7f6b18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b18004800 0x7f6b18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b340cf110 0x7f6b180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34163440 con 0x7f6b180039b0 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b340b7b30 con 0x7f6b18004800 2024-04-15T15:20:48.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1800b9c0 con 0x7f6b340cf110 2024-04-15T15:20:48.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b31e00640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b180039b0 0x7f6b18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b31400640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b18004800 0x7f6b18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b32800640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b340cf110 0x7f6b180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b31e00640 1 -- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b340cf110 msgr2=0x7f6b180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.884+0000 7f6b31e00640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b340cf110 0x7f6b180051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.885+0000 7f6b31e00640 1 -- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b18004800 msgr2=0x7f6b18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:48.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.885+0000 7f6b31e00640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b18004800 0x7f6b18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.885+0000 7f6b31e00640 1 -- 172.21.15.49:0/2064351074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b18003520 con 0x7f6b180039b0 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.886+0000 7f6b31e00640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b180039b0 0x7f6b18003dd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400c3e0 tx=0x7f6b18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153b2e25ebb9206a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.886+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400c0e0 con 0x7f6b180039b0 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.886+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b24030070 con 0x7f6b180039b0 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.886+0000 7f6b30a00640 1 -- 172.21.15.49:0/2064351074 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b240058d0 con 0x7f6b180039b0 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.887+0000 7f6b33200640 1 -- 172.21.15.49:0/2064351074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b180039b0 msgr2=0x7f6b18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.887+0000 7f6b33200640 1 --2- 172.21.15.49:0/2064351074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b180039b0 0x7f6b18003dd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400c3e0 tx=0x7f6b18005830 comp rx=0 tx=0).stop 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.887+0000 7f6b32800640 1 -- 172.21.15.49:0/2064351074 reap_dead start 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.887+0000 7f6b33200640 1 -- 172.21.15.49:0/2064351074 shutdown_connections 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.887+0000 7f6b33200640 1 -- 172.21.15.49:0/2064351074 >> 172.21.15.49:0/2064351074 conn(0x7f6b3406cf00 msgr2=0x7f6b3406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.887+0000 7f6b33200640 1 -- 172.21.15.49:0/2064351074 shutdown_connections 2024-04-15T15:20:48.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.887+0000 7f6b33200640 1 -- 172.21.15.49:0/2064351074 wait complete. 2024-04-15T15:20:48.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.889+0000 7f6b33200640 1 Processor -- start 2024-04-15T15:20:48.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.889+0000 7f6b33200640 1 -- start start 2024-04-15T15:20:48.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b33200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b180039b0 0x7f6b3414e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b33200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b340cd590 0x7f6b340cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b33200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b341506c0 0x7f6b34150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b33200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b340d2b90 con 0x7f6b340cd590 2024-04-15T15:20:48.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b33200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b340cf880 con 0x7f6b180039b0 2024-04-15T15:20:48.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b33200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6b340d2e70 con 0x7f6b341506c0 2024-04-15T15:20:48.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b31400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b340cd590 0x7f6b340cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b31400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b340cd590 0x7f6b340cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56788/0 (socket says 172.21.15.49:56788) 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b31400640 1 -- 172.21.15.49:0/1846408483 learned_addr learned my addr 172.21.15.49:0/1846408483 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b31e00640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b180039b0 0x7f6b3414e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b31400640 1 -- 172.21.15.49:0/1846408483 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b341506c0 msgr2=0x7f6b34150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.890+0000 7f6b32800640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b341506c0 0x7f6b34150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.891+0000 7f6b31400640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b341506c0 0x7f6b34150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.891+0000 7f6b31400640 1 -- 172.21.15.49:0/1846408483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b180039b0 msgr2=0x7f6b3414e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.891+0000 7f6b31400640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b180039b0 0x7f6b3414e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.891+0000 7f6b31400640 1 -- 172.21.15.49:0/1846408483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b340ce6c0 con 0x7f6b340cd590 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.891+0000 7f6b31e00640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b180039b0 0x7f6b3414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.891+0000 7f6b31400640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b340cd590 0x7f6b340cda10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6b20011600 tx=0x7f6b20011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.891+0000 7f6b2a000640 1 -- 172.21.15.49:0/1846408483 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000f920 con 0x7f6b340cd590 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.891+0000 7f6b2a000640 1 -- 172.21.15.49:0/1846408483 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2000fac0 con 0x7f6b340cd590 2024-04-15T15:20:48.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.891+0000 7f6b2a000640 1 -- 172.21.15.49:0/1846408483 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000fe70 con 0x7f6b340cd590 2024-04-15T15:20:48.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.891+0000 7f6b33200640 1 -- 172.21.15.49:0/1846408483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3415e610 con 0x7f6b340cd590 2024-04-15T15:20:48.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.892+0000 7f6b33200640 1 -- 172.21.15.49:0/1846408483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3415ebd0 con 0x7f6b340cd590 2024-04-15T15:20:48.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.893+0000 7f6b33200640 1 -- 172.21.15.49:0/1846408483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b340755c0 con 0x7f6b340cd590 2024-04-15T15:20:48.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.894+0000 7f6b2a000640 1 -- 172.21.15.49:0/1846408483 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6b20015460 con 0x7f6b340cd590 2024-04-15T15:20:48.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.895+0000 7f6b2a000640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6b1406c980 0x7f6b1406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:48.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.895+0000 7f6b2a000640 1 -- 172.21.15.49:0/1846408483 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f6b2008c530 con 0x7f6b340cd590 2024-04-15T15:20:48.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.895+0000 7f6b31e00640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6b1406c980 0x7f6b1406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:48.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.896+0000 7f6b31e00640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6b1406c980 0x7f6b1406ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6b24034000 tx=0x7f6b24007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:48.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:48.900+0000 7f6b2a000640 1 -- 172.21.15.49:0/1846408483 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b20019030 con 0x7f6b340cd590 2024-04-15T15:20:49.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.106+0000 7f6b33200640 1 -- 172.21.15.49:0/1846408483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b341616d0 con 0x7f6b340cd590 2024-04-15T15:20:49.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.107+0000 7f6b2a000640 1 -- 172.21.15.49:0/1846408483 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16798 (secure 0 0 0) 0x7f6b2005a8a0 con 0x7f6b340cd590 2024-04-15T15:20:49.110 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:49.110 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:28.618835+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6837/1023024717":"2024-04-16T15:20:28.618766+0000","172.21.15.49:6836/1023024717":"2024-04-16T15:20:28.618766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:49.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.110+0000 7f6b33200640 1 -- 172.21.15.49:0/1846408483 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6b1406c980 msgr2=0x7f6b1406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.110+0000 7f6b33200640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6b1406c980 0x7f6b1406ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6b24034000 tx=0x7f6b24007000 comp rx=0 tx=0).stop 2024-04-15T15:20:49.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.110+0000 7f6b33200640 1 -- 172.21.15.49:0/1846408483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b340cd590 msgr2=0x7f6b340cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.110+0000 7f6b33200640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b340cd590 0x7f6b340cda10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6b20011600 tx=0x7f6b20011ac0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.110+0000 7f6b33200640 1 -- 172.21.15.49:0/1846408483 shutdown_connections 2024-04-15T15:20:49.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.110+0000 7f6b33200640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f6b1406c980 0x7f6b1406ee40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.110+0000 7f6b33200640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b341506c0 0x7f6b34150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.110+0000 7f6b33200640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b340cd590 0x7f6b340cda10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.111+0000 7f6b33200640 1 --2- 172.21.15.49:0/1846408483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b180039b0 0x7f6b3414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.111+0000 7f6b33200640 1 -- 172.21.15.49:0/1846408483 >> 172.21.15.49:0/1846408483 conn(0x7f6b3406cf00 msgr2=0x7f6b340b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:49.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.111+0000 7f6b33200640 1 -- 172.21.15.49:0/1846408483 shutdown_connections 2024-04-15T15:20:49.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.111+0000 7f6b33200640 1 -- 172.21.15.49:0/1846408483 wait complete. 2024-04-15T15:20:49.133 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-15T15:20:49.133 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:49.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.363+0000 7f7b72400640 1 Processor -- start 2024-04-15T15:20:49.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.363+0000 7f7b72400640 1 -- start start 2024-04-15T15:20:49.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.364+0000 7f7b72400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7b6c1461f0 0x7f7b6c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.364+0000 7f7b72400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b6c0b4be0 con 0x7f7b6c0b3fe0 2024-04-15T15:20:49.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.364+0000 7f7b72400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b6c0b4620 con 0x7f7b6c0cedf0 2024-04-15T15:20:49.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.364+0000 7f7b72400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f7b6c0a9640 con 0x7f7b6c1461f0 2024-04-15T15:20:49.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.365+0000 7f7b6be00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7b6c0b3fe0 0x7f7b6c15d7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56600/0 (socket says 172.21.15.49:56600) 2024-04-15T15:20:49.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.365+0000 7f7b6be00640 1 -- 172.21.15.49:0/3103194814 learned_addr learned my addr 172.21.15.49:0/3103194814 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:49.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.365+0000 7f7b6b400640 1 --2- 172.21.15.49:0/3103194814 >> v2:172.21.15.107:3300/0 conn(0x7f7b6c1461f0 0x7f7b6c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.365+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1928932853 0 0) 0x7f7b6c0b4be0 con 0x7f7b6c0b3fe0 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.365+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b50003520 con 0x7f7b6c0b3fe0 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1057271058 0 0) 0x7f7b50003520 con 0x7f7b6c0b3fe0 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 >> v2:172.21.15.107:3300/0 conn(0x7f7b6c1461f0 msgr2=0x7f7b6c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 --2- 172.21.15.49:0/3103194814 >> v2:172.21.15.107:3300/0 conn(0x7f7b6c1461f0 0x7f7b6c0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 >> v1:172.21.15.107:6789/0 conn(0x7f7b6c0cedf0 legacy=0x7f7b6c0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c0b4900 con 0x7f7b6c0b3fe0 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f7b640019f0 con 0x7f7b6c0b3fe0 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 >> v1:172.21.15.49:6789/0 conn(0x7f7b6c0b3fe0 legacy=0x7f7b6c15d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 --2- 172.21.15.49:0/3103194814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b50003a00 0x7f7b50003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 --2- 172.21.15.49:0/3103194814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 --2- 172.21.15.49:0/3103194814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b500051c0 0x7f7b5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5000c090 con 0x7f7b50004800 2024-04-15T15:20:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5000c240 con 0x7f7b50003a00 2024-04-15T15:20:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5000c440 con 0x7f7b500051c0 2024-04-15T15:20:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.366+0000 7f7b62a00640 1 --2- 172.21.15.49:0/3103194814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.367+0000 7f7b6b400640 1 --2- 172.21.15.49:0/3103194814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b50003a00 0x7f7b50003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.367+0000 7f7b62a00640 1 -- 172.21.15.49:0/3103194814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b500051c0 msgr2=0x7f7b5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.367+0000 7f7b62a00640 1 --2- 172.21.15.49:0/3103194814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b500051c0 0x7f7b5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.367+0000 7f7b62a00640 1 -- 172.21.15.49:0/3103194814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b50003a00 msgr2=0x7f7b50003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.367+0000 7f7b62a00640 1 --2- 172.21.15.49:0/3103194814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b50003a00 0x7f7b50003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.367+0000 7f7b62a00640 1 -- 172.21.15.49:0/3103194814 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b50003520 con 0x7f7b50004800 2024-04-15T15:20:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.367+0000 7f7b6be00640 1 -- 172.21.15.49:0/3103194814 reap_dead start 2024-04-15T15:20:49.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.368+0000 7f7b62a00640 1 --2- 172.21.15.49:0/3103194814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b50004c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7b54001750 tx=0x7f7b50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fc67e23020ae31 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.368+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54001370 con 0x7f7b50004800 2024-04-15T15:20:49.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.368+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b540052d0 con 0x7f7b50004800 2024-04-15T15:20:49.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.368+0000 7f7b6aa00640 1 -- 172.21.15.49:0/3103194814 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54003830 con 0x7f7b50004800 2024-04-15T15:20:49.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.369+0000 7f7b72400640 1 -- 172.21.15.49:0/3103194814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 msgr2=0x7f7b50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.369+0000 7f7b72400640 1 --2- 172.21.15.49:0/3103194814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b50004c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7b54001750 tx=0x7f7b50005830 comp rx=0 tx=0).stop 2024-04-15T15:20:49.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.369+0000 7f7b72400640 1 -- 172.21.15.49:0/3103194814 shutdown_connections 2024-04-15T15:20:49.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.369+0000 7f7b72400640 1 --2- 172.21.15.49:0/3103194814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b50004c80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.369+0000 7f7b72400640 1 -- 172.21.15.49:0/3103194814 >> 172.21.15.49:0/3103194814 conn(0x7f7b6c0b27b0 msgr2=0x7f7b6c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:49.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.369+0000 7f7b72400640 1 -- 172.21.15.49:0/3103194814 shutdown_connections 2024-04-15T15:20:49.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.369+0000 7f7b72400640 1 -- 172.21.15.49:0/3103194814 wait complete. 2024-04-15T15:20:49.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.371+0000 7f7b72400640 1 Processor -- start 2024-04-15T15:20:49.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.371+0000 7f7b72400640 1 -- start start 2024-04-15T15:20:49.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.372+0000 7f7b72400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b6c0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.372+0000 7f7b72400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b6c141f80 0x7f7b6c15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.372+0000 7f7b72400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c142400 0x7f7b6c15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.372+0000 7f7b72400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c15d290 con 0x7f7b50004800 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.372+0000 7f7b72400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c1465d0 con 0x7f7b6c142400 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.372+0000 7f7b72400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c0ce8b0 con 0x7f7b6c141f80 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.372+0000 7f7b6b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b6c0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.372+0000 7f7b6b400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b6c0ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56798/0 (socket says 172.21.15.49:56798) 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.372+0000 7f7b6b400640 1 -- 172.21.15.49:0/4149682255 learned_addr learned my addr 172.21.15.49:0/4149682255 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.373+0000 7f7b62a00640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b6c141f80 0x7f7b6c15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.373+0000 7f7b6be00640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c142400 0x7f7b6c15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.373+0000 7f7b6b400640 1 -- 172.21.15.49:0/4149682255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b6c141f80 msgr2=0x7f7b6c15bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.373+0000 7f7b6b400640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b6c141f80 0x7f7b6c15bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.373+0000 7f7b6b400640 1 -- 172.21.15.49:0/4149682255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c142400 msgr2=0x7f7b6c15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.373+0000 7f7b6b400640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c142400 0x7f7b6c15c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.373+0000 7f7b6b400640 1 -- 172.21.15.49:0/4149682255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c1e5a50 con 0x7f7b50004800 2024-04-15T15:20:49.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.373+0000 7f7b6b400640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b6c0ae3f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c011510 tx=0x7f7b5c0119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.374+0000 7f7b63e00640 1 -- 172.21.15.49:0/4149682255 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c015870 con 0x7f7b50004800 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.374+0000 7f7b63e00640 1 -- 172.21.15.49:0/4149682255 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c015a10 con 0x7f7b50004800 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.374+0000 7f7b63e00640 1 -- 172.21.15.49:0/4149682255 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c015dc0 con 0x7f7b50004800 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.374+0000 7f7b72400640 1 -- 172.21.15.49:0/4149682255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c1e5d30 con 0x7f7b50004800 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.374+0000 7f7b72400640 1 -- 172.21.15.49:0/4149682255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c06e650 con 0x7f7b50004800 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.375+0000 7f7b72400640 1 -- 172.21.15.49:0/4149682255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c068c80 con 0x7f7b50004800 2024-04-15T15:20:49.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.376+0000 7f7b63e00640 1 -- 172.21.15.49:0/4149682255 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7b5c013400 con 0x7f7b50004800 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.377+0000 7f7b63e00640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b4806c980 0x7f7b4806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.377+0000 7f7b63e00640 1 -- 172.21.15.49:0/4149682255 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f7b5c08c5c0 con 0x7f7b50004800 2024-04-15T15:20:49.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.377+0000 7f7b62a00640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b4806c980 0x7f7b4806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.378+0000 7f7b62a00640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b4806c980 0x7f7b4806ee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7b54033000 tx=0x7f7b54009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.381+0000 7f7b63e00640 1 -- 172.21.15.49:0/4149682255 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b5c019030 con 0x7f7b50004800 2024-04-15T15:20:49.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.619+0000 7f7b72400640 1 -- 172.21.15.49:0/4149682255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b6c15c6e0 con 0x7f7b50004800 2024-04-15T15:20:49.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.620+0000 7f7b63e00640 1 -- 172.21.15.49:0/4149682255 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5188 (secure 0 0 0) 0x7f7b5c05a930 con 0x7f7b50004800 2024-04-15T15:20:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:49.622 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:00.037472+0000","modified":"2024-04-15T15:20:32.648230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/3168788676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":3168788676},{"type":"v1","addr":"172.21.15.49:6837","nonce":3168788676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5445,"qdb_cluster":[5445]},"id":2}]} 2024-04-15T15:20:49.622 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-04-15T15:20:49.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 -- 172.21.15.49:0/4149682255 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b4806c980 msgr2=0x7f7b4806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b4806c980 0x7f7b4806ee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7b54033000 tx=0x7f7b54009030 comp rx=0 tx=0).stop 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 -- 172.21.15.49:0/4149682255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 msgr2=0x7f7b6c0ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b6c0ae3f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c011510 tx=0x7f7b5c0119d0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 -- 172.21.15.49:0/4149682255 shutdown_connections 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f7b4806c980 0x7f7b4806ee40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b6c141f80 0x7f7b6c15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c142400 0x7f7b6c15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 --2- 172.21.15.49:0/4149682255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004800 0x7f7b6c0ae3f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 -- 172.21.15.49:0/4149682255 >> 172.21.15.49:0/4149682255 conn(0x7f7b6c0b27b0 msgr2=0x7f7b6c15de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 -- 172.21.15.49:0/4149682255 shutdown_connections 2024-04-15T15:20:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.623+0000 7f7b72400640 1 -- 172.21.15.49:0/4149682255 wait complete. 2024-04-15T15:20:49.642 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:20:49.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.864+0000 7fc2abe00640 1 Processor -- start 2024-04-15T15:20:49.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.864+0000 7fc2abe00640 1 -- start start 2024-04-15T15:20:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.866+0000 7fc2abe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ac15b860 0x7fc2ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.866+0000 7fc2abe00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ac0cdfb0 con 0x7fc2ac06f320 2024-04-15T15:20:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.866+0000 7fc2abe00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ac15f2b0 con 0x7fc2ac15b480 2024-04-15T15:20:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.866+0000 7fc2abe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac15f060 con 0x7fc2ac15b860 2024-04-15T15:20:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.866+0000 7fc2ab400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc2ac06f320 0x7fc2ac0b3fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56612/0 (socket says 172.21.15.49:56612) 2024-04-15T15:20:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.866+0000 7fc2ab400640 1 -- 172.21.15.49:0/2701997017 learned_addr learned my addr 172.21.15.49:0/2701997017 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.867+0000 7fc2aa000640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ac15b860 0x7fc2ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.867+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 968823413 0 0) 0x7fc2ac0cdfb0 con 0x7fc2ac06f320 2024-04-15T15:20:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.867+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc290003520 con 0x7fc2ac06f320 2024-04-15T15:20:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.867+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1986360726 0 0) 0x7fc290003520 con 0x7fc2ac06f320 2024-04-15T15:20:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.867+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ac15b860 msgr2=0x7fc2ac06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.867+0000 7fc2a9600640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ac15b860 0x7fc2ac06ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.867+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 >> v1:172.21.15.107:6789/0 conn(0x7fc2ac15b480 legacy=0x7fc2ac0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.867+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac15f590 con 0x7fc2ac06f320 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.867+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fc2a40019f0 con 0x7fc2ac06f320 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 >> v1:172.21.15.49:6789/0 conn(0x7fc2ac06f320 legacy=0x7fc2ac0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2a9600640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2900039e0 0x7fc290003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2a9600640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc290004800 0x7fc290004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2a9600640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ac15b860 0x7fc2900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac15f060 con 0x7fc2900039e0 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc29000baa0 con 0x7fc290004800 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc29000bca0 con 0x7fc2ac15b860 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2aaa00640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2900039e0 0x7fc290003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2aa000640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc290004800 0x7fc290004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2aaa00640 1 -- 172.21.15.49:0/2701997017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ac15b860 msgr2=0x7fc2900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:49.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2aaa00640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ac15b860 0x7fc2900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2aaa00640 1 -- 172.21.15.49:0/2701997017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc290004800 msgr2=0x7fc290004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:49.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2aaa00640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc290004800 0x7fc290004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.868+0000 7fc2aaa00640 1 -- 172.21.15.49:0/2701997017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc290003520 con 0x7fc2900039e0 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.870+0000 7fc2aaa00640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2900039e0 0x7fc290003e00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc29c001720 tx=0x7fc290005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88cb7c8babd3f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.870+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c001370 con 0x7fc2900039e0 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.870+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc29c0052d0 con 0x7fc2900039e0 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.870+0000 7fc2a9600640 1 -- 172.21.15.49:0/2701997017 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c003830 con 0x7fc2900039e0 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.870+0000 7fc2abe00640 1 -- 172.21.15.49:0/2701997017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2900039e0 msgr2=0x7fc290003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.870+0000 7fc2abe00640 1 --2- 172.21.15.49:0/2701997017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2900039e0 0x7fc290003e00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc29c001720 tx=0x7fc290005830 comp rx=0 tx=0).stop 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.870+0000 7fc2ab400640 1 -- 172.21.15.49:0/2701997017 reap_dead start 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.871+0000 7fc2abe00640 1 -- 172.21.15.49:0/2701997017 shutdown_connections 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.871+0000 7fc2abe00640 1 -- 172.21.15.49:0/2701997017 >> 172.21.15.49:0/2701997017 conn(0x7fc2ac06b930 msgr2=0x7fc2ac06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.871+0000 7fc2abe00640 1 -- 172.21.15.49:0/2701997017 shutdown_connections 2024-04-15T15:20:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.871+0000 7fc2abe00640 1 -- 172.21.15.49:0/2701997017 wait complete. 2024-04-15T15:20:49.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.872+0000 7fc2abe00640 1 Processor -- start 2024-04-15T15:20:49.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.873+0000 7fc2abe00640 1 -- start start 2024-04-15T15:20:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.873+0000 7fc2abe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2900039e0 0x7fc2ac152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.873+0000 7fc2abe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac14a490 0x7fc2ac150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.873+0000 7fc2abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ac14c760 0x7fc2ac14cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.873+0000 7fc2abe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac15fec0 con 0x7fc2ac14a490 2024-04-15T15:20:49.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2abe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac06f770 con 0x7fc2ac14c760 2024-04-15T15:20:49.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2abe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac15fb50 con 0x7fc2900039e0 2024-04-15T15:20:49.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2aa000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac14a490 0x7fc2ac150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2aa000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac14a490 0x7fc2ac150080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56824/0 (socket says 172.21.15.49:56824) 2024-04-15T15:20:49.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2aa000640 1 -- 172.21.15.49:0/2863643048 learned_addr learned my addr 172.21.15.49:0/2863643048 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:49.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2ab400640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ac14c760 0x7fc2ac14cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2aaa00640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2900039e0 0x7fc2ac152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2aa000640 1 -- 172.21.15.49:0/2863643048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2900039e0 msgr2=0x7fc2ac152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2aa000640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2900039e0 0x7fc2ac152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2aa000640 1 -- 172.21.15.49:0/2863643048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ac14c760 msgr2=0x7fc2ac14cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:49.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2aa000640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ac14c760 0x7fc2ac14cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:49.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2aa000640 1 -- 172.21.15.49:0/2863643048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac14b4c0 con 0x7fc2ac14a490 2024-04-15T15:20:49.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.874+0000 7fc2ab400640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ac14c760 0x7fc2ac14cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:49.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.875+0000 7fc2aa000640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac14a490 0x7fc2ac150080 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc298011560 tx=0x7fc298011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.875+0000 7fc2a2a00640 1 -- 172.21.15.49:0/2863643048 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298004220 con 0x7fc2ac14a490 2024-04-15T15:20:49.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.875+0000 7fc2a2a00640 1 -- 172.21.15.49:0/2863643048 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2980043c0 con 0x7fc2ac14a490 2024-04-15T15:20:49.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.875+0000 7fc2a2a00640 1 -- 172.21.15.49:0/2863643048 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298015870 con 0x7fc2ac14a490 2024-04-15T15:20:49.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.875+0000 7fc2abe00640 1 -- 172.21.15.49:0/2863643048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac14ac20 con 0x7fc2ac14a490 2024-04-15T15:20:49.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.875+0000 7fc2abe00640 1 -- 172.21.15.49:0/2863643048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac14b1e0 con 0x7fc2ac14a490 2024-04-15T15:20:49.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.877+0000 7fc2abe00640 1 -- 172.21.15.49:0/2863643048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac0755c0 con 0x7fc2ac14a490 2024-04-15T15:20:49.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.878+0000 7fc2a2a00640 1 -- 172.21.15.49:0/2863643048 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc298015a10 con 0x7fc2ac14a490 2024-04-15T15:20:49.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.878+0000 7fc2a2a00640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc28c06c9b0 0x7fc28c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:49.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.879+0000 7fc2a2a00640 1 -- 172.21.15.49:0/2863643048 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fc29808c180 con 0x7fc2ac14a490 2024-04-15T15:20:49.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.879+0000 7fc2aaa00640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc28c06c9b0 0x7fc28c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:49.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.880+0000 7fc2aaa00640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc28c06c9b0 0x7fc28c06ee70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc29c033000 tx=0x7fc29c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:49.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:49.883+0000 7fc2a2a00640 1 -- 172.21.15.49:0/2863643048 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc298019030 con 0x7fc2ac14a490 2024-04-15T15:20:50.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.123+0000 7fc2abe00640 1 -- 172.21.15.49:0/2863643048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2ac0b3930 con 0x7fc2ac14a490 2024-04-15T15:20:50.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.124+0000 7fc2a2a00640 1 -- 172.21.15.49:0/2863643048 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5188 (secure 0 0 0) 0x7fc29805a530 con 0x7fc2ac14a490 2024-04-15T15:20:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:50.126 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:20:00.037472+0000","modified":"2024-04-15T15:20:32.648230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/3168788676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":3168788676},{"type":"v1","addr":"172.21.15.49:6837","nonce":3168788676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5445,"qdb_cluster":[5445]},"id":2}]} 2024-04-15T15:20:50.126 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-04-15T15:20:50.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.126+0000 7fc2abe00640 1 -- 172.21.15.49:0/2863643048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc28c06c9b0 msgr2=0x7fc28c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:50.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.126+0000 7fc2abe00640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc28c06c9b0 0x7fc28c06ee70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc29c033000 tx=0x7fc29c009030 comp rx=0 tx=0).stop 2024-04-15T15:20:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.126+0000 7fc2abe00640 1 -- 172.21.15.49:0/2863643048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac14a490 msgr2=0x7fc2ac150080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.126+0000 7fc2abe00640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac14a490 0x7fc2ac150080 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc298011560 tx=0x7fc298011a20 comp rx=0 tx=0).stop 2024-04-15T15:20:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.126+0000 7fc2abe00640 1 -- 172.21.15.49:0/2863643048 shutdown_connections 2024-04-15T15:20:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.126+0000 7fc2abe00640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc28c06c9b0 0x7fc28c06ee70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.126+0000 7fc2abe00640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ac14c760 0x7fc2ac14cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.126+0000 7fc2abe00640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac14a490 0x7fc2ac150080 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.126+0000 7fc2abe00640 1 --2- 172.21.15.49:0/2863643048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2900039e0 0x7fc2ac152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.126+0000 7fc2abe00640 1 -- 172.21.15.49:0/2863643048 >> 172.21.15.49:0/2863643048 conn(0x7fc2ac06b930 msgr2=0x7fc2ac14e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.127+0000 7fc2abe00640 1 -- 172.21.15.49:0/2863643048 shutdown_connections 2024-04-15T15:20:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.127+0000 7fc2abe00640 1 -- 172.21.15.49:0/2863643048 wait complete. 2024-04-15T15:20:50.146 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:50.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.370+0000 7ff94d400640 1 Processor -- start 2024-04-15T15:20:50.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.370+0000 7ff94d400640 1 -- start start 2024-04-15T15:20:50.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.372+0000 7ff94d400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff948072660 0x7ff94806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.372+0000 7ff94d400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff94815cbe0 0x7ff94815d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.372+0000 7ff94d400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff94815dbb0 con 0x7ff9480cee00 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.372+0000 7ff94d400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff94806f4f0 con 0x7ff948072660 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.372+0000 7ff94d400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff94815d8d0 con 0x7ff94815cbe0 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.372+0000 7ff947400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff948072660 0x7ff94806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.372+0000 7ff947400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff948072660 0x7ff94806e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56826/0 (socket says 172.21.15.49:56826) 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.372+0000 7ff947400640 1 -- 172.21.15.49:0/637483002 learned_addr learned my addr 172.21.15.49:0/637483002 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.373+0000 7ff94ca00640 1 --2- 172.21.15.49:0/637483002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff94815cbe0 0x7ff94815d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.373+0000 7ff947400640 1 -- 172.21.15.49:0/637483002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff94815cbe0 msgr2=0x7ff94815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.373+0000 7ff947400640 1 --2- 172.21.15.49:0/637483002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff94815cbe0 0x7ff94815d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.373+0000 7ff947400640 1 -- 172.21.15.49:0/637483002 >> v1:172.21.15.107:6789/0 conn(0x7ff9480cee00 legacy=0x7ff948071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.373+0000 7ff947400640 1 -- 172.21.15.49:0/637483002 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94806dde0 con 0x7ff948072660 2024-04-15T15:20:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.373+0000 7ff94ca00640 1 --2- 172.21.15.49:0/637483002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff94815cbe0 0x7ff94815d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.374+0000 7ff947400640 1 --2- 172.21.15.49:0/637483002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff948072660 0x7ff94806e8c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff934009870 tx=0x7ff93402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344dd488f242438d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.374+0000 7ff946a00640 1 -- 172.21.15.49:0/637483002 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934035020 con 0x7ff948072660 2024-04-15T15:20:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.374+0000 7ff946a00640 1 -- 172.21.15.49:0/637483002 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff934031640 con 0x7ff948072660 2024-04-15T15:20:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.374+0000 7ff946a00640 1 -- 172.21.15.49:0/637483002 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93402d040 con 0x7ff948072660 2024-04-15T15:20:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.375+0000 7ff94d400640 1 -- 172.21.15.49:0/637483002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff948072660 msgr2=0x7ff94806e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.375+0000 7ff94d400640 1 --2- 172.21.15.49:0/637483002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff948072660 0x7ff94806e8c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff934009870 tx=0x7ff93402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.375+0000 7ff94d400640 1 -- 172.21.15.49:0/637483002 shutdown_connections 2024-04-15T15:20:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.375+0000 7ff94d400640 1 --2- 172.21.15.49:0/637483002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff94815cbe0 0x7ff94815d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.375+0000 7ff94d400640 1 --2- 172.21.15.49:0/637483002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff948072660 0x7ff94806e8c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.375+0000 7ff94d400640 1 -- 172.21.15.49:0/637483002 >> 172.21.15.49:0/637483002 conn(0x7ff94806b930 msgr2=0x7ff94806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.375+0000 7ff94d400640 1 -- 172.21.15.49:0/637483002 shutdown_connections 2024-04-15T15:20:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.375+0000 7ff94d400640 1 -- 172.21.15.49:0/637483002 wait complete. 2024-04-15T15:20:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.377+0000 7ff94d400640 1 Processor -- start 2024-04-15T15:20:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.377+0000 7ff94d400640 1 -- start start 2024-04-15T15:20:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.378+0000 7ff94d400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff948072660 0x7ff9481e6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.378+0000 7ff94d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9480cee00 0x7ff948159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.378+0000 7ff94d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff94815cbe0 0x7ff94815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.378+0000 7ff94d400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff94815d5b0 con 0x7ff9480cee00 2024-04-15T15:20:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.378+0000 7ff94d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff94806e3a0 con 0x7ff94815cbe0 2024-04-15T15:20:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.378+0000 7ff94d400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff94815dda0 con 0x7ff948072660 2024-04-15T15:20:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.378+0000 7ff947400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9480cee00 0x7ff948159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.378+0000 7ff947400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9480cee00 0x7ff948159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56834/0 (socket says 172.21.15.49:56834) 2024-04-15T15:20:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.379+0000 7ff947400640 1 -- 172.21.15.49:0/734526175 learned_addr learned my addr 172.21.15.49:0/734526175 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:50.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.379+0000 7ff947400640 1 -- 172.21.15.49:0/734526175 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff948072660 msgr2=0x7ff9481e6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.379+0000 7ff947400640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff948072660 0x7ff9481e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.379+0000 7ff947400640 1 -- 172.21.15.49:0/734526175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff94815cbe0 msgr2=0x7ff94815bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.379+0000 7ff947400640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff94815cbe0 0x7ff94815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.379+0000 7ff947400640 1 -- 172.21.15.49:0/734526175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff948158d60 con 0x7ff9480cee00 2024-04-15T15:20:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.379+0000 7ff947400640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9480cee00 0x7ff948159e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff93402b930 tx=0x7ff9340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.380+0000 7ff93fe00640 1 -- 172.21.15.49:0/734526175 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934040070 con 0x7ff9480cee00 2024-04-15T15:20:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.380+0000 7ff93fe00640 1 -- 172.21.15.49:0/734526175 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff934039c10 con 0x7ff9480cee00 2024-04-15T15:20:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.380+0000 7ff93fe00640 1 -- 172.21.15.49:0/734526175 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934035040 con 0x7ff9480cee00 2024-04-15T15:20:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.380+0000 7ff94d400640 1 -- 172.21.15.49:0/734526175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff948159040 con 0x7ff9480cee00 2024-04-15T15:20:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.380+0000 7ff94d400640 1 -- 172.21.15.49:0/734526175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff948159600 con 0x7ff9480cee00 2024-04-15T15:20:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.381+0000 7ff94d400640 1 -- 172.21.15.49:0/734526175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9480755c0 con 0x7ff9480cee00 2024-04-15T15:20:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.382+0000 7ff93fe00640 1 -- 172.21.15.49:0/734526175 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff93404b060 con 0x7ff9480cee00 2024-04-15T15:20:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.383+0000 7ff93fe00640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff92806c980 0x7ff92806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.384+0000 7ff93fe00640 1 -- 172.21.15.49:0/734526175 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7ff9340ad830 con 0x7ff9480cee00 2024-04-15T15:20:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.384+0000 7ff947e00640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff92806c980 0x7ff92806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.384+0000 7ff947e00640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff92806c980 0x7ff92806ee40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff938003cc0 tx=0x7ff9380079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:50.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.388+0000 7ff93fe00640 1 -- 172.21.15.49:0/734526175 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff93402f300 con 0x7ff9480cee00 2024-04-15T15:20:50.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.596+0000 7ff94d400640 1 -- 172.21.15.49:0/734526175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff948068a60 con 0x7ff9480cee00 2024-04-15T15:20:50.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.597+0000 7ff93fe00640 1 -- 172.21.15.49:0/734526175 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16798 (secure 0 0 0) 0x7ff93407bb80 con 0x7ff9480cee00 2024-04-15T15:20:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:50.600 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:28.618835+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:19:58.063498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:19:59.009342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6837/1023024717":"2024-04-16T15:20:28.618766+0000","172.21.15.49:6836/1023024717":"2024-04-16T15:20:28.618766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:50.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 -- 172.21.15.49:0/734526175 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff92806c980 msgr2=0x7ff92806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:50.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff92806c980 0x7ff92806ee40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff938003cc0 tx=0x7ff9380079b0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 -- 172.21.15.49:0/734526175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9480cee00 msgr2=0x7ff948159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9480cee00 0x7ff948159e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff93402b930 tx=0x7ff9340046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 -- 172.21.15.49:0/734526175 shutdown_connections 2024-04-15T15:20:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff92806c980 0x7ff92806ee40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff94815cbe0 0x7ff94815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9480cee00 0x7ff948159e00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 --2- 172.21.15.49:0/734526175 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff948072660 0x7ff9481e6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 -- 172.21.15.49:0/734526175 >> 172.21.15.49:0/734526175 conn(0x7ff94806b930 msgr2=0x7ff948071f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.600+0000 7ff94d400640 1 -- 172.21.15.49:0/734526175 shutdown_connections 2024-04-15T15:20:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.601+0000 7ff94d400640 1 -- 172.21.15.49:0/734526175 wait complete. 2024-04-15T15:20:50.623 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-15T15:20:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.845+0000 7fd29f200640 1 Processor -- start 2024-04-15T15:20:50.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.846+0000 7fd29f200640 1 -- start start 2024-04-15T15:20:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.847+0000 7fd29f200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd2a015eab0 0x7fd2a0162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.847+0000 7fd29f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 0x7fd2a00d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.847+0000 7fd29f200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd2a00d1a90 0x7fd2a00b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.847+0000 7fd29f200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a00d2e50 con 0x7fd2a00d1a90 2024-04-15T15:20:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.847+0000 7fd29f200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a00b78f0 con 0x7fd2a015eab0 2024-04-15T15:20:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.847+0000 7fd29f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a0163010 con 0x7fd2a0162b70 2024-04-15T15:20:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd29e800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd2a00d1a90 0x7fd2a00b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd29e800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd2a00d1a90 0x7fd2a00b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56838/0 (socket says 172.21.15.49:56838) 2024-04-15T15:20:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd29e800640 1 -- 172.21.15.49:0/2965279827 learned_addr learned my addr 172.21.15.49:0/2965279827 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:50.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd297e00640 1 --2- 172.21.15.49:0/2965279827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 0x7fd2a00d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd29de00640 1 --2- 172.21.15.49:0/2965279827 >> v2:172.21.15.107:3300/0 conn(0x7fd2a015eab0 0x7fd2a0162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd29e800640 1 -- 172.21.15.49:0/2965279827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 msgr2=0x7fd2a00d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:50.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd29e800640 1 --2- 172.21.15.49:0/2965279827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 0x7fd2a00d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd29e800640 1 -- 172.21.15.49:0/2965279827 >> v2:172.21.15.107:3300/0 conn(0x7fd2a015eab0 msgr2=0x7fd2a0162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:50.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd29e800640 1 --2- 172.21.15.49:0/2965279827 >> v2:172.21.15.107:3300/0 conn(0x7fd2a015eab0 0x7fd2a0162630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd29e800640 1 -- 172.21.15.49:0/2965279827 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a01632f0 con 0x7fd2a00d1a90 2024-04-15T15:20:50.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.848+0000 7fd297e00640 1 --2- 172.21.15.49:0/2965279827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 0x7fd2a00d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:50.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.849+0000 7fd29e800640 1 --2- 172.21.15.49:0/2965279827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a00d1a90 0x7fd2a00b6240 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd2a008d2c0 tx=0x7fd29802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14adba3f6086abb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.850+0000 7fd29d400640 1 -- 172.21.15.49:0/2965279827 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29802ca70 con 0x7fd2a00d1a90 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.850+0000 7fd29d400640 1 -- 172.21.15.49:0/2965279827 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd29802cc10 con 0x7fd2a00d1a90 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.850+0000 7fd29d400640 1 -- 172.21.15.49:0/2965279827 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2980343f0 con 0x7fd2a00d1a90 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.850+0000 7fd29f200640 1 -- 172.21.15.49:0/2965279827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a00d1a90 msgr2=0x7fd2a00b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.850+0000 7fd29f200640 1 --2- 172.21.15.49:0/2965279827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a00d1a90 0x7fd2a00b6240 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd2a008d2c0 tx=0x7fd29802bed0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.851+0000 7fd29f200640 1 -- 172.21.15.49:0/2965279827 shutdown_connections 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.851+0000 7fd29f200640 1 --2- 172.21.15.49:0/2965279827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a00d1a90 0x7fd2a00b6240 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.851+0000 7fd29f200640 1 --2- 172.21.15.49:0/2965279827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 0x7fd2a00d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.851+0000 7fd29f200640 1 --2- 172.21.15.49:0/2965279827 >> v2:172.21.15.107:3300/0 conn(0x7fd2a015eab0 0x7fd2a0162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.851+0000 7fd29f200640 1 -- 172.21.15.49:0/2965279827 >> 172.21.15.49:0/2965279827 conn(0x7fd2a0073240 msgr2=0x7fd2a0073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.851+0000 7fd29f200640 1 -- 172.21.15.49:0/2965279827 shutdown_connections 2024-04-15T15:20:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.851+0000 7fd29f200640 1 -- 172.21.15.49:0/2965279827 wait complete. 2024-04-15T15:20:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.852+0000 7fd29f200640 1 Processor -- start 2024-04-15T15:20:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.853+0000 7fd29f200640 1 -- start start 2024-04-15T15:20:50.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd29f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a00d1a90 0x7fd2a0161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd29f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a015eab0 0x7fd2a00b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd29f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 0x7fd2a0152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd29f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a00ba590 con 0x7fd2a015eab0 2024-04-15T15:20:50.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd29f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a015ee90 con 0x7fd2a00d1a90 2024-04-15T15:20:50.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd29f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a00ba710 con 0x7fd2a0162b70 2024-04-15T15:20:50.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd297e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a015eab0 0x7fd2a00b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd297e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a015eab0 0x7fd2a00b8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56842/0 (socket says 172.21.15.49:56842) 2024-04-15T15:20:50.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd297e00640 1 -- 172.21.15.49:0/3891136063 learned_addr learned my addr 172.21.15.49:0/3891136063 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:50.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd29de00640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a00d1a90 0x7fd2a0161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd297e00640 1 -- 172.21.15.49:0/3891136063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 msgr2=0x7fd2a0152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:50.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.854+0000 7fd29e800640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 0x7fd2a0152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd297e00640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 0x7fd2a0152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd297e00640 1 -- 172.21.15.49:0/3891136063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a00d1a90 msgr2=0x7fd2a0161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:50.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd297e00640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a00d1a90 0x7fd2a0161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:50.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd297e00640 1 -- 172.21.15.49:0/3891136063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a015b6a0 con 0x7fd2a015eab0 2024-04-15T15:20:50.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd29de00640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a00d1a90 0x7fd2a0161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:50.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd297e00640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a015eab0 0x7fd2a00b8ec0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c7f0 tx=0x7fd28800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:50.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd296000640 1 -- 172.21.15.49:0/3891136063 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288015070 con 0x7fd2a015eab0 2024-04-15T15:20:50.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd296000640 1 -- 172.21.15.49:0/3891136063 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd288007820 con 0x7fd2a015eab0 2024-04-15T15:20:50.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd296000640 1 -- 172.21.15.49:0/3891136063 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288007bd0 con 0x7fd2a015eab0 2024-04-15T15:20:50.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd29f200640 1 -- 172.21.15.49:0/3891136063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a015ab80 con 0x7fd2a015eab0 2024-04-15T15:20:50.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.855+0000 7fd29f200640 1 -- 172.21.15.49:0/3891136063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a015b3c0 con 0x7fd2a015eab0 2024-04-15T15:20:50.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.856+0000 7fd29f200640 1 -- 172.21.15.49:0/3891136063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a00755c0 con 0x7fd2a015eab0 2024-04-15T15:20:50.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.857+0000 7fd296000640 1 -- 172.21.15.49:0/3891136063 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd288018070 con 0x7fd2a015eab0 2024-04-15T15:20:50.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.859+0000 7fd296000640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd27c06c980 0x7fd27c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:50.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.859+0000 7fd296000640 1 -- 172.21.15.49:0/3891136063 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fd28808bf00 con 0x7fd2a015eab0 2024-04-15T15:20:50.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.859+0000 7fd29de00640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd27c06c980 0x7fd27c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:50.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.860+0000 7fd29de00640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd27c06c980 0x7fd27c06ee40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd290006fd0 tx=0x7fd290008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:50.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:50.864+0000 7fd296000640 1 -- 172.21.15.49:0/3891136063 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2a015ab80 con 0x7fd2a015eab0 2024-04-15T15:20:51.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.106+0000 7fd29f200640 1 -- 172.21.15.49:0/3891136063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd2a0153c40 con 0x7fd2a015eab0 2024-04-15T15:20:51.109 INFO:tasks.ceph.mon.a.smithi049.stderr:2024-04-15T15:20:51.108+0000 7fbf18400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-15T15:20:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.112+0000 7fd296000640 1 -- 172.21.15.49:0/3891136063 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7fd28805a0d0 con 0x7fd2a015eab0 2024-04-15T15:20:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.115+0000 7fd29f200640 1 -- 172.21.15.49:0/3891136063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd27c06c980 msgr2=0x7fd27c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.115+0000 7fd29f200640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd27c06c980 0x7fd27c06ee40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd290006fd0 tx=0x7fd290008040 comp rx=0 tx=0).stop 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.115+0000 7fd29f200640 1 -- 172.21.15.49:0/3891136063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a015eab0 msgr2=0x7fd2a00b8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.115+0000 7fd29f200640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a015eab0 0x7fd2a00b8ec0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c7f0 tx=0x7fd28800ccb0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.116+0000 7fd29f200640 1 -- 172.21.15.49:0/3891136063 shutdown_connections 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.116+0000 7fd29f200640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd27c06c980 0x7fd27c06ee40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.116+0000 7fd29f200640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2a0162b70 0x7fd2a0152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.116+0000 7fd29f200640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a015eab0 0x7fd2a00b8ec0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.116+0000 7fd29f200640 1 --2- 172.21.15.49:0/3891136063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a00d1a90 0x7fd2a0161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.116+0000 7fd29f200640 1 -- 172.21.15.49:0/3891136063 >> 172.21.15.49:0/3891136063 conn(0x7fd2a0073240 msgr2=0x7fd2a00c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.116+0000 7fd29f200640 1 -- 172.21.15.49:0/3891136063 shutdown_connections 2024-04-15T15:20:51.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.116+0000 7fd29f200640 1 -- 172.21.15.49:0/3891136063 wait complete. 2024-04-15T15:20:51.129 INFO:tasks.ceph.mds.c.smithi049.stderr: -2847> 2024-04-15T15:20:34.640+0000 7f4225800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:20:51.139 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-15T15:20:51.193 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2024-04-15T15:20:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.328+0000 7f580b000640 1 Processor -- start 2024-04-15T15:20:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.328+0000 7f580b000640 1 -- start start 2024-04-15T15:20:51.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.329+0000 7f580b000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f58040725f0 0x7f58040ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:51.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.330+0000 7f580b000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58040cd2d0 con 0x7f58040ce9e0 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.330+0000 7f580b000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58040cd5b0 con 0x7f58041461c0 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.330+0000 7f580b000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f58040cf1d0 con 0x7f58040725f0 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.330+0000 7f580a600640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f58040ce9e0 0x7f580406df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56638/0 (socket says 172.21.15.49:56638) 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.330+0000 7f580a600640 1 -- 172.21.15.49:0/3585637877 learned_addr learned my addr 172.21.15.49:0/3585637877 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.330+0000 7f5809200640 1 --2- 172.21.15.49:0/3585637877 >> v2:172.21.15.107:3300/0 conn(0x7f58040725f0 0x7f58040ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.330+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1966873556 0 0) 0x7f58040cd2d0 con 0x7f58040ce9e0 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.330+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57ec003520 con 0x7f58040ce9e0 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3382814896 0 0) 0x7f57ec003520 con 0x7f58040ce9e0 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 >> v2:172.21.15.107:3300/0 conn(0x7f58040725f0 msgr2=0x7f58040ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 --2- 172.21.15.49:0/3585637877 >> v2:172.21.15.107:3300/0 conn(0x7f58040725f0 0x7f58040ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 >> v1:172.21.15.107:6789/0 conn(0x7f58041461c0 legacy=0x7f5804071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58040cd890 con 0x7f58040ce9e0 2024-04-15T15:20:51.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f57fc0019f0 con 0x7f58040ce9e0 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 >> v1:172.21.15.49:6789/0 conn(0x7f58040ce9e0 legacy=0x7f580406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 --2- 172.21.15.49:0/3585637877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57ec0039b0 0x7f57ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 --2- 172.21.15.49:0/3585637877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f57ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 --2- 172.21.15.49:0/3585637877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57ec0051c0 0x7f57ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec00c030 con 0x7f57ec004800 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec00c1e0 con 0x7f57ec0051c0 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f57ec00c3e0 con 0x7f57ec0039b0 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.331+0000 7f5809200640 1 --2- 172.21.15.49:0/3585637877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f57ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.332+0000 7f5809200640 1 -- 172.21.15.49:0/3585637877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57ec0039b0 msgr2=0x7f57ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.332+0000 7f5809200640 1 --2- 172.21.15.49:0/3585637877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57ec0039b0 0x7f57ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.332+0000 7f5809200640 1 -- 172.21.15.49:0/3585637877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57ec0051c0 msgr2=0x7f57ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.332+0000 7f5809200640 1 --2- 172.21.15.49:0/3585637877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57ec0051c0 0x7f57ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.332+0000 7f5809200640 1 -- 172.21.15.49:0/3585637877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec003520 con 0x7f57ec004800 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.332+0000 7f580a600640 1 -- 172.21.15.49:0/3585637877 reap_dead start 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.333+0000 7f5809200640 1 --2- 172.21.15.49:0/3585637877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f57ec004c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009ac0 tx=0x7f57ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49317ee57c77164 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.333+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0039070 con 0x7f57ec004800 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.333+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f002c6b0 con 0x7f57ec004800 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.333+0000 7f5803e00640 1 -- 172.21.15.49:0/3585637877 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0009690 con 0x7f57ec004800 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.334+0000 7f580b000640 1 -- 172.21.15.49:0/3585637877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 msgr2=0x7f57ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.334+0000 7f580b000640 1 --2- 172.21.15.49:0/3585637877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f57ec004c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009ac0 tx=0x7f57ec005830 comp rx=0 tx=0).stop 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.334+0000 7f580b000640 1 -- 172.21.15.49:0/3585637877 shutdown_connections 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.334+0000 7f580b000640 1 --2- 172.21.15.49:0/3585637877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f57ec004c80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.334+0000 7f580b000640 1 -- 172.21.15.49:0/3585637877 >> 172.21.15.49:0/3585637877 conn(0x7f580406b930 msgr2=0x7f580406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.334+0000 7f580b000640 1 -- 172.21.15.49:0/3585637877 shutdown_connections 2024-04-15T15:20:51.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.334+0000 7f580b000640 1 -- 172.21.15.49:0/3585637877 wait complete. 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.335+0000 7f580b000640 1 Processor -- start 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.336+0000 7f580b000640 1 -- start start 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.336+0000 7f580b000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f58040b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f580b000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5804156710 0x7f58041594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f580b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f580415ad70 0x7f58040b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f580b000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58040cee30 con 0x7f57ec004800 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f580b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58040cd080 con 0x7f580415ad70 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f580b000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f58040cefb0 con 0x7f5804156710 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f5809c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f58040b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f5809c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f58040b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56886/0 (socket says 172.21.15.49:56886) 2024-04-15T15:20:51.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f5809c00640 1 -- 172.21.15.49:0/2176778286 learned_addr learned my addr 172.21.15.49:0/2176778286 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f5809200640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5804156710 0x7f58041594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f580a600640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f580415ad70 0x7f58040b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f5809c00640 1 -- 172.21.15.49:0/2176778286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5804156710 msgr2=0x7f58041594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f5809c00640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5804156710 0x7f58041594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f5809c00640 1 -- 172.21.15.49:0/2176778286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f580415ad70 msgr2=0x7f58040b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f5809c00640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f580415ad70 0x7f58040b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f5809c00640 1 -- 172.21.15.49:0/2176778286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5804159cd0 con 0x7f57ec004800 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.337+0000 7f580a600640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f580415ad70 0x7f58040b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.338+0000 7f5809c00640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f58040b5460 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c430 tx=0x7f57f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.338+0000 7f5801600640 1 -- 172.21.15.49:0/2176778286 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8017070 con 0x7f57ec004800 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.338+0000 7f5801600640 1 -- 172.21.15.49:0/2176778286 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f80052f0 con 0x7f57ec004800 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.338+0000 7f580b000640 1 -- 172.21.15.49:0/2176778286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5804159fb0 con 0x7f57ec004800 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.338+0000 7f5801600640 1 -- 172.21.15.49:0/2176778286 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8003830 con 0x7f57ec004800 2024-04-15T15:20:51.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.338+0000 7f580b000640 1 -- 172.21.15.49:0/2176778286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580415a570 con 0x7f57ec004800 2024-04-15T15:20:51.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.339+0000 7f580b000640 1 -- 172.21.15.49:0/2176778286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58040755c0 con 0x7f57ec004800 2024-04-15T15:20:51.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.341+0000 7f5801600640 1 -- 172.21.15.49:0/2176778286 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f57f801a070 con 0x7f57ec004800 2024-04-15T15:20:51.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.342+0000 7f5801600640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f57e406c9b0 0x7f57e406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:51.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.343+0000 7f5801600640 1 -- 172.21.15.49:0/2176778286 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5881+0+0 (secure 0 0 0) 0x7f57f808b100 con 0x7f57ec004800 2024-04-15T15:20:51.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.343+0000 7f5809200640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f57e406c9b0 0x7f57e406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:51.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.344+0000 7f5809200640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f57e406c9b0 0x7f57e406ee70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f57f0036000 tx=0x7f57f0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:51.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.347+0000 7f5801600640 1 -- 172.21.15.49:0/2176778286 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57f8059390 con 0x7f57ec004800 2024-04-15T15:20:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:51.585+0000 7f580b000640 1 -- 172.21.15.49:0/2176778286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5804072e40 con 0x7f57ec004800 2024-04-15T15:20:52.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.115+0000 7f5801600640 1 -- 172.21.15.49:0/2176778286 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f57f8058c90 con 0x7f57ec004800 2024-04-15T15:20:52.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.119+0000 7f580b000640 1 -- 172.21.15.49:0/2176778286 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f57e406c9b0 msgr2=0x7f57e406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:52.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.119+0000 7f580b000640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f57e406c9b0 0x7f57e406ee70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f57f0036000 tx=0x7f57f0030000 comp rx=0 tx=0).stop 2024-04-15T15:20:52.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.119+0000 7f580b000640 1 -- 172.21.15.49:0/2176778286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 msgr2=0x7f58040b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:52.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.119+0000 7f580b000640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f58040b5460 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c430 tx=0x7f57f800c8f0 comp rx=0 tx=0).stop 2024-04-15T15:20:52.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.119+0000 7f580b000640 1 -- 172.21.15.49:0/2176778286 shutdown_connections 2024-04-15T15:20:52.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.119+0000 7f580b000640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f57e406c9b0 0x7f57e406ee70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:52.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.119+0000 7f580b000640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5804156710 0x7f58041594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:52.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.119+0000 7f580b000640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f580415ad70 0x7f58040b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.119+0000 7f580b000640 1 --2- 172.21.15.49:0/2176778286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57ec004800 0x7f58040b5460 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.119+0000 7f580b000640 1 -- 172.21.15.49:0/2176778286 >> 172.21.15.49:0/2176778286 conn(0x7f580406b930 msgr2=0x7f5804145270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.120+0000 7f580b000640 1 -- 172.21.15.49:0/2176778286 shutdown_connections 2024-04-15T15:20:52.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.120+0000 7f580b000640 1 -- 172.21.15.49:0/2176778286 wait complete. 2024-04-15T15:20:52.142 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-15T15:20:52.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.363+0000 7f1924a00640 1 Processor -- start 2024-04-15T15:20:52.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.363+0000 7f1924a00640 1 -- start start 2024-04-15T15:20:52.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.365+0000 7f1924a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19200cec60 0x7f1920071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.365+0000 7f1924a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f192015d420 0x7f192006f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.365+0000 7f1924a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192013fc70 con 0x7f1920072450 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.365+0000 7f1924a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f192013f9e0 con 0x7f192015d420 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.365+0000 7f1924a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1920072890 con 0x7f19200cec60 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.365+0000 7f191f200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f192015d420 0x7f192006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.365+0000 7f191f200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f192015d420 0x7f192006f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56900/0 (socket says 172.21.15.49:56900) 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.365+0000 7f191f200640 1 -- 172.21.15.49:0/1588887418 learned_addr learned my addr 172.21.15.49:0/1588887418 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.366+0000 7f191e800640 1 --2- 172.21.15.49:0/1588887418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19200cec60 0x7f1920071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.366+0000 7f191f200640 1 -- 172.21.15.49:0/1588887418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19200cec60 msgr2=0x7f1920071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.366+0000 7f191f200640 1 --2- 172.21.15.49:0/1588887418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19200cec60 0x7f1920071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.366+0000 7f191f200640 1 -- 172.21.15.49:0/1588887418 >> v1:172.21.15.107:6789/0 conn(0x7f1920072450 legacy=0x7f192015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:52.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.366+0000 7f191f200640 1 -- 172.21.15.49:0/1588887418 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1920141a00 con 0x7f192015d420 2024-04-15T15:20:52.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.367+0000 7f191f200640 1 --2- 172.21.15.49:0/1588887418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192015d420 0x7f192006f110 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1918009e50 tx=0x7f191802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b140ca816484cd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:52.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.367+0000 7f191d400640 1 -- 172.21.15.49:0/1588887418 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191802cb00 con 0x7f192015d420 2024-04-15T15:20:52.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.367+0000 7f191d400640 1 -- 172.21.15.49:0/1588887418 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f191802cca0 con 0x7f192015d420 2024-04-15T15:20:52.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.367+0000 7f191d400640 1 -- 172.21.15.49:0/1588887418 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918034610 con 0x7f192015d420 2024-04-15T15:20:52.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.368+0000 7f1924a00640 1 -- 172.21.15.49:0/1588887418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192015d420 msgr2=0x7f192006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:52.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.368+0000 7f1924a00640 1 --2- 172.21.15.49:0/1588887418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192015d420 0x7f192006f110 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1918009e50 tx=0x7f191802bf60 comp rx=0 tx=0).stop 2024-04-15T15:20:52.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.368+0000 7f1924a00640 1 -- 172.21.15.49:0/1588887418 shutdown_connections 2024-04-15T15:20:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.368+0000 7f1924a00640 1 --2- 172.21.15.49:0/1588887418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192015d420 0x7f192006f110 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.368+0000 7f1924a00640 1 --2- 172.21.15.49:0/1588887418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19200cec60 0x7f1920071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.368+0000 7f1924a00640 1 -- 172.21.15.49:0/1588887418 >> 172.21.15.49:0/1588887418 conn(0x7f19200b6170 msgr2=0x7f192006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.369+0000 7f1924a00640 1 -- 172.21.15.49:0/1588887418 shutdown_connections 2024-04-15T15:20:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.369+0000 7f1924a00640 1 -- 172.21.15.49:0/1588887418 wait complete. 2024-04-15T15:20:52.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.371+0000 7f1924a00640 1 Processor -- start 2024-04-15T15:20:52.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.371+0000 7f1924a00640 1 -- start start 2024-04-15T15:20:52.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.372+0000 7f1924a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1920072450 0x7f1920152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:52.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.372+0000 7f1924a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19200cec60 0x7f19201f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:52.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.372+0000 7f1924a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192015d420 0x7f19201f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:52.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.372+0000 7f1924a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f19201401f0 con 0x7f19200cec60 2024-04-15T15:20:52.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.372+0000 7f1924a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f192013ff10 con 0x7f192015d420 2024-04-15T15:20:52.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.372+0000 7f1924a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f19201404d0 con 0x7f1920072450 2024-04-15T15:20:52.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.373+0000 7f191de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19200cec60 0x7f19201f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:52.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.373+0000 7f191de00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19200cec60 0x7f19201f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56914/0 (socket says 172.21.15.49:56914) 2024-04-15T15:20:52.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.373+0000 7f191de00640 1 -- 172.21.15.49:0/2405798417 learned_addr learned my addr 172.21.15.49:0/2405798417 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.373+0000 7f191e800640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1920072450 0x7f1920152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.373+0000 7f191f200640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192015d420 0x7f19201f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.373+0000 7f191de00640 1 -- 172.21.15.49:0/2405798417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1920072450 msgr2=0x7f1920152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.373+0000 7f191de00640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1920072450 0x7f1920152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.373+0000 7f191de00640 1 -- 172.21.15.49:0/2405798417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192015d420 msgr2=0x7f19201f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.373+0000 7f191de00640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192015d420 0x7f19201f2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.373+0000 7f191de00640 1 -- 172.21.15.49:0/2405798417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19200b1390 con 0x7f19200cec60 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.374+0000 7f191de00640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19200cec60 0x7f19201f25f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f190c00b3f0 tx=0x7f190c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.374+0000 7f1916a00640 1 -- 172.21.15.49:0/2405798417 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c015020 con 0x7f19200cec60 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.374+0000 7f1916a00640 1 -- 172.21.15.49:0/2405798417 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f190c011830 con 0x7f19200cec60 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.374+0000 7f1924a00640 1 -- 172.21.15.49:0/2405798417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192013d220 con 0x7f19200cec60 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.374+0000 7f1916a00640 1 -- 172.21.15.49:0/2405798417 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c011c20 con 0x7f19200cec60 2024-04-15T15:20:52.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.374+0000 7f1924a00640 1 -- 172.21.15.49:0/2405798417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1920159fd0 con 0x7f19200cec60 2024-04-15T15:20:52.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.375+0000 7f1924a00640 1 -- 172.21.15.49:0/2405798417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1920068b40 con 0x7f19200cec60 2024-04-15T15:20:52.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.376+0000 7f1916a00640 1 -- 172.21.15.49:0/2405798417 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f190c018020 con 0x7f19200cec60 2024-04-15T15:20:52.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.378+0000 7f1916a00640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f190006c9b0 0x7f190006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:52.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.378+0000 7f1916a00640 1 -- 172.21.15.49:0/2405798417 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5881+0+0 (secure 0 0 0) 0x7f190c08b760 con 0x7f19200cec60 2024-04-15T15:20:52.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.378+0000 7f191e800640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f190006c9b0 0x7f190006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:52.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.379+0000 7f191e800640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f190006c9b0 0x7f190006ee70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f191000a970 tx=0x7f1910008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:52.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.382+0000 7f1916a00640 1 -- 172.21.15.49:0/2405798417 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f190c01f030 con 0x7f19200cec60 2024-04-15T15:20:52.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:52.586+0000 7f1924a00640 1 -- 172.21.15.49:0/2405798417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19200730e0 con 0x7f19200cec60 2024-04-15T15:20:53.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.119+0000 7f1916a00640 1 -- 172.21.15.49:0/2405798417 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) ==== 181+0+0 (secure 0 0 0) 0x7f190c0172c0 con 0x7f19200cec60 2024-04-15T15:20:53.122 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2024-04-15T15:20:53.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.123+0000 7f1924a00640 1 -- 172.21.15.49:0/2405798417 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f190006c9b0 msgr2=0x7f190006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:53.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.123+0000 7f1924a00640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f190006c9b0 0x7f190006ee70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f191000a970 tx=0x7f1910008040 comp rx=0 tx=0).stop 2024-04-15T15:20:53.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.123+0000 7f1924a00640 1 -- 172.21.15.49:0/2405798417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19200cec60 msgr2=0x7f19201f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:53.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.123+0000 7f1924a00640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19200cec60 0x7f19201f25f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f190c00b3f0 tx=0x7f190c00b8b0 comp rx=0 tx=0).stop 2024-04-15T15:20:53.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.123+0000 7f1924a00640 1 -- 172.21.15.49:0/2405798417 shutdown_connections 2024-04-15T15:20:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.123+0000 7f1924a00640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f190006c9b0 0x7f190006ee70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.123+0000 7f1924a00640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192015d420 0x7f19201f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.123+0000 7f1924a00640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19200cec60 0x7f19201f25f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.123+0000 7f1924a00640 1 --2- 172.21.15.49:0/2405798417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1920072450 0x7f1920152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.123+0000 7f1924a00640 1 -- 172.21.15.49:0/2405798417 >> 172.21.15.49:0/2405798417 conn(0x7f19200b6170 msgr2=0x7f192006de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.124+0000 7f1924a00640 1 -- 172.21.15.49:0/2405798417 shutdown_connections 2024-04-15T15:20:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.124+0000 7f1924a00640 1 -- 172.21.15.49:0/2405798417 wait complete. 2024-04-15T15:20:53.150 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-15T15:20:53.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.377+0000 7f867f200640 1 Processor -- start 2024-04-15T15:20:53.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.378+0000 7f867f200640 1 -- start start 2024-04-15T15:20:53.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.379+0000 7f867f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86800cf2c0 0x7f868006de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:53.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.379+0000 7f867f200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868006e380 con 0x7f868006f230 2024-04-15T15:20:53.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.379+0000 7f867f200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86800b37f0 con 0x7f8680071f10 2024-04-15T15:20:53.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.379+0000 7f867f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f868006eea0 con 0x7f86800cf2c0 2024-04-15T15:20:53.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.380+0000 7f867e800640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f868006f230 0x7f86800b3cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56648/0 (socket says 172.21.15.49:56648) 2024-04-15T15:20:53.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.380+0000 7f867e800640 1 -- 172.21.15.49:0/3358154441 learned_addr learned my addr 172.21.15.49:0/3358154441 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:53.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.380+0000 7f867de00640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86800cf2c0 0x7f868006de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:53.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.380+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079290602 0 0) 0x7f8678001490 con 0x7f868006f230 2024-04-15T15:20:53.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.380+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8664003520 con 0x7f868006f230 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.380+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3877065623 0 0) 0x7f8664003520 con 0x7f868006f230 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.380+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86800cf2c0 msgr2=0x7f868006de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.380+0000 7f867ca00640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86800cf2c0 0x7f868006de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.380+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 >> v1:172.21.15.107:6789/0 conn(0x7f8680071f10 legacy=0x7f86800ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86801630e0 con 0x7f868006f230 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f8678001bc0 con 0x7f868006f230 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 >> v1:172.21.15.49:6789/0 conn(0x7f868006f230 legacy=0x7f86800b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867ca00640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86800cf2c0 0x7f86640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867ca00640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8664004040 0x7f8664004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867ca00640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664004d40 0x7f86640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f868006eea0 con 0x7f8664004040 2024-04-15T15:20:53.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f866400bbb0 con 0x7f8664004d40 2024-04-15T15:20:53.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f866400bdb0 con 0x7f86800cf2c0 2024-04-15T15:20:53.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867d400640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8664004040 0x7f8664004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:53.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867de00640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86800cf2c0 0x7f86640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:53.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.381+0000 7f867e800640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664004d40 0x7f86640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:53.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.382+0000 7f867d400640 1 -- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86800cf2c0 msgr2=0x7f86640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:53.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.382+0000 7f867d400640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86800cf2c0 0x7f86640039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:53.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.382+0000 7f867d400640 1 -- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664004d40 msgr2=0x7f86640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:53.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.382+0000 7f867d400640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664004d40 0x7f86640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:53.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.382+0000 7f867d400640 1 -- 172.21.15.49:0/3358154441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8664003520 con 0x7f8664004040 2024-04-15T15:20:53.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.382+0000 7f867e800640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8664004d40 0x7f86640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:53.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.383+0000 7f867d400640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8664004040 0x7f8664004800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f866c003db0 tx=0x7f8664005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0c0cf5fa21676c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:53.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.383+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c037070 con 0x7f8664004040 2024-04-15T15:20:53.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.383+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c032600 con 0x7f8664004040 2024-04-15T15:20:53.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.383+0000 7f867ca00640 1 -- 172.21.15.49:0/3358154441 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c003980 con 0x7f8664004040 2024-04-15T15:20:53.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.384+0000 7f867f200640 1 -- 172.21.15.49:0/3358154441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8664004040 msgr2=0x7f8664004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:53.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.384+0000 7f867f200640 1 --2- 172.21.15.49:0/3358154441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8664004040 0x7f8664004800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f866c003db0 tx=0x7f8664005830 comp rx=0 tx=0).stop 2024-04-15T15:20:53.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.384+0000 7f867e800640 1 -- 172.21.15.49:0/3358154441 reap_dead start 2024-04-15T15:20:53.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.384+0000 7f867f200640 1 -- 172.21.15.49:0/3358154441 shutdown_connections 2024-04-15T15:20:53.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.384+0000 7f867f200640 1 -- 172.21.15.49:0/3358154441 >> 172.21.15.49:0/3358154441 conn(0x7f86800b6170 msgr2=0x7f868006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:53.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.384+0000 7f867f200640 1 -- 172.21.15.49:0/3358154441 shutdown_connections 2024-04-15T15:20:53.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.384+0000 7f867f200640 1 -- 172.21.15.49:0/3358154441 wait complete. 2024-04-15T15:20:53.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.386+0000 7f867f200640 1 Processor -- start 2024-04-15T15:20:53.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.386+0000 7f867f200640 1 -- start start 2024-04-15T15:20:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867f200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8664004040 0x7f868014d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867f200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f868014d560 0x7f8680074d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8680075270 0x7f868014e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867f200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f868006ebc0 con 0x7f868014d560 2024-04-15T15:20:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f868006e8e0 con 0x7f8680075270 2024-04-15T15:20:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867f200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f86800cf6a0 con 0x7f8664004040 2024-04-15T15:20:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f868014d560 0x7f8680074d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f868014d560 0x7f8680074d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56924/0 (socket says 172.21.15.49:56924) 2024-04-15T15:20:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867d400640 1 -- 172.21.15.49:0/2754795355 learned_addr learned my addr 172.21.15.49:0/2754795355 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867de00640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8664004040 0x7f868014d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.387+0000 7f867e800640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8680075270 0x7f868014e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f867d400640 1 -- 172.21.15.49:0/2754795355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8664004040 msgr2=0x7f868014d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f867d400640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8664004040 0x7f868014d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f867d400640 1 -- 172.21.15.49:0/2754795355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8680075270 msgr2=0x7f868014e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f867d400640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8680075270 0x7f868014e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f867d400640 1 -- 172.21.15.49:0/2754795355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8680151040 con 0x7f868014d560 2024-04-15T15:20:53.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f867de00640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8664004040 0x7f868014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:53.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f867d400640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f868014d560 0x7f8680074d30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f866c003ed0 tx=0x7f866c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:53.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f8676000640 1 -- 172.21.15.49:0/2754795355 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c037070 con 0x7f868014d560 2024-04-15T15:20:53.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f8676000640 1 -- 172.21.15.49:0/2754795355 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c038040 con 0x7f868014d560 2024-04-15T15:20:53.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f8676000640 1 -- 172.21.15.49:0/2754795355 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c007a70 con 0x7f868014d560 2024-04-15T15:20:53.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.388+0000 7f867f200640 1 -- 172.21.15.49:0/2754795355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868014f570 con 0x7f868014d560 2024-04-15T15:20:53.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.389+0000 7f867f200640 1 -- 172.21.15.49:0/2754795355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86801515c0 con 0x7f868014d560 2024-04-15T15:20:53.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.390+0000 7f867f200640 1 -- 172.21.15.49:0/2754795355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8680068b40 con 0x7f868014d560 2024-04-15T15:20:53.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.391+0000 7f8676000640 1 -- 172.21.15.49:0/2754795355 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f866c032600 con 0x7f868014d560 2024-04-15T15:20:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.392+0000 7f8676000640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f866006c980 0x7f866006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.392+0000 7f8676000640 1 -- 172.21.15.49:0/2754795355 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5443+0+0 (secure 0 0 0) 0x7f866c0adc40 con 0x7f868014d560 2024-04-15T15:20:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.392+0000 7f867de00640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f866006c980 0x7f866006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.393+0000 7f867de00640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f866006c980 0x7f866006ee40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8670006fd0 tx=0x7f8670008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:53.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.396+0000 7f8676000640 1 -- 172.21.15.49:0/2754795355 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f866c07bf70 con 0x7f868014d560 2024-04-15T15:20:53.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:53.620+0000 7f867f200640 1 -- 172.21.15.49:0/2754795355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8680163790 con 0x7f868014d560 2024-04-15T15:20:54.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.123+0000 7f8676000640 1 -- 172.21.15.49:0/2754795355 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) ==== 169+0+0 (secure 0 0 0) 0x7f866c07b870 con 0x7f868014d560 2024-04-15T15:20:54.125 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2024-04-15T15:20:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 -- 172.21.15.49:0/2754795355 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f866006c980 msgr2=0x7f866006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f866006c980 0x7f866006ee40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8670006fd0 tx=0x7f8670008040 comp rx=0 tx=0).stop 2024-04-15T15:20:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 -- 172.21.15.49:0/2754795355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f868014d560 msgr2=0x7f8680074d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f868014d560 0x7f8680074d30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f866c003ed0 tx=0x7f866c0036f0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 -- 172.21.15.49:0/2754795355 shutdown_connections 2024-04-15T15:20:54.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f866006c980 0x7f866006ee40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8680075270 0x7f868014e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f868014d560 0x7f8680074d30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 --2- 172.21.15.49:0/2754795355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8664004040 0x7f868014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 -- 172.21.15.49:0/2754795355 >> 172.21.15.49:0/2754795355 conn(0x7f86800b6170 msgr2=0x7f8680165410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:54.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.127+0000 7f867f200640 1 -- 172.21.15.49:0/2754795355 shutdown_connections 2024-04-15T15:20:54.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.128+0000 7f867f200640 1 -- 172.21.15.49:0/2754795355 wait complete. 2024-04-15T15:20:54.153 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_quiesce_splitauth 2024-04-15T15:20:54.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.374+0000 7f8d71800640 1 Processor -- start 2024-04-15T15:20:54.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.374+0000 7f8d71800640 1 -- start start 2024-04-15T15:20:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.376+0000 7f8d71800640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8d6c0b2c30 0x7f8d6c0728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.376+0000 7f8d71800640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6c06e660 con 0x7f8d6c06f420 2024-04-15T15:20:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.376+0000 7f8d71800640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6c06e940 con 0x7f8d6c146240 2024-04-15T15:20:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.376+0000 7f8d71800640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f8d6c146620 con 0x7f8d6c0b2c30 2024-04-15T15:20:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.377+0000 7f8d6b400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8d6c06f420 0x7f8d6c0721a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42462/0 (socket says 172.21.15.49:42462) 2024-04-15T15:20:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.377+0000 7f8d6b400640 1 -- 172.21.15.49:0/2302055366 learned_addr learned my addr 172.21.15.49:0/2302055366 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.377+0000 7f8d70e00640 1 --2- 172.21.15.49:0/2302055366 >> v2:172.21.15.107:3300/0 conn(0x7f8d6c0b2c30 0x7f8d6c0728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.377+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2149341588 0 0) 0x7f8d6c06e660 con 0x7f8d6c06f420 2024-04-15T15:20:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.377+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d50003520 con 0x7f8d6c06f420 2024-04-15T15:20:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2357650101 0 0) 0x7f8d50003520 con 0x7f8d6c06f420 2024-04-15T15:20:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 >> v2:172.21.15.107:3300/0 conn(0x7f8d6c0b2c30 msgr2=0x7f8d6c0728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 --2- 172.21.15.49:0/2302055366 >> v2:172.21.15.107:3300/0 conn(0x7f8d6c0b2c30 0x7f8d6c0728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 >> v1:172.21.15.107:6789/0 conn(0x7f8d6c146240 legacy=0x7f8d6c0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c06ec20 con 0x7f8d6c06f420 2024-04-15T15:20:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f8d58001670 con 0x7f8d6c06f420 2024-04-15T15:20:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 >> v1:172.21.15.49:6789/0 conn(0x7f8d6c06f420 legacy=0x7f8d6c0721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d50003a00 0x7f8d50003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d50004800 0x7f8d50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c146620 con 0x7f8d500051c0 2024-04-15T15:20:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5000c0c0 con 0x7f8d50004800 2024-04-15T15:20:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5000c2c0 con 0x7f8d50003a00 2024-04-15T15:20:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.378+0000 7f8d70e00640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.379+0000 7f8d6be00640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d50003a00 0x7f8d50003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.379+0000 7f8d6b400640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d50004800 0x7f8d50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.379+0000 7f8d70e00640 1 -- 172.21.15.49:0/2302055366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d50003a00 msgr2=0x7f8d50003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.379+0000 7f8d70e00640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d50003a00 0x7f8d50003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.379+0000 7f8d70e00640 1 -- 172.21.15.49:0/2302055366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d50004800 msgr2=0x7f8d50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.379+0000 7f8d70e00640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d50004800 0x7f8d50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.379+0000 7f8d70e00640 1 -- 172.21.15.49:0/2302055366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d50003520 con 0x7f8d500051c0 2024-04-15T15:20:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.380+0000 7f8d70e00640 1 -- 172.21.15.49:0/2302055366 reap_dead start 2024-04-15T15:20:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.380+0000 7f8d70e00640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d5000b9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009fc0 tx=0x7f8d50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9c65dfc35ae6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.380+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64009c10 con 0x7f8d500051c0 2024-04-15T15:20:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.381+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6402cca0 con 0x7f8d500051c0 2024-04-15T15:20:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.381+0000 7f8d6aa00640 1 -- 172.21.15.49:0/2302055366 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64003bd0 con 0x7f8d500051c0 2024-04-15T15:20:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.381+0000 7f8d71800640 1 -- 172.21.15.49:0/2302055366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 msgr2=0x7f8d5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.381+0000 7f8d71800640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d5000b9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009fc0 tx=0x7f8d50005830 comp rx=0 tx=0).stop 2024-04-15T15:20:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.381+0000 7f8d71800640 1 -- 172.21.15.49:0/2302055366 shutdown_connections 2024-04-15T15:20:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.381+0000 7f8d71800640 1 --2- 172.21.15.49:0/2302055366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d5000b9c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.381+0000 7f8d71800640 1 -- 172.21.15.49:0/2302055366 >> 172.21.15.49:0/2302055366 conn(0x7f8d6c06b930 msgr2=0x7f8d6c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.381+0000 7f8d71800640 1 -- 172.21.15.49:0/2302055366 shutdown_connections 2024-04-15T15:20:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.381+0000 7f8d71800640 1 -- 172.21.15.49:0/2302055366 wait complete. 2024-04-15T15:20:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.383+0000 7f8d71800640 1 Processor -- start 2024-04-15T15:20:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.383+0000 7f8d71800640 1 -- start start 2024-04-15T15:20:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.384+0000 7f8d71800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d6c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.384+0000 7f8d71800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d6c0d0120 0x7f8d6c0d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.384+0000 7f8d71800640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d6c0c5680 0x7f8d6c0c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.384+0000 7f8d71800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c0b26a0 con 0x7f8d500051c0 2024-04-15T15:20:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.384+0000 7f8d71800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c0b2520 con 0x7f8d6c0d0120 2024-04-15T15:20:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.384+0000 7f8d71800640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6c0b2820 con 0x7f8d6c0c5680 2024-04-15T15:20:54.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d6be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d6c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d6be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d6c15c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43506/0 (socket says 172.21.15.49:43506) 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d6be00640 1 -- 172.21.15.49:0/2155221488 learned_addr learned my addr 172.21.15.49:0/2155221488 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d70e00640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d6c0c5680 0x7f8d6c0c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d6b400640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d6c0d0120 0x7f8d6c0d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d6be00640 1 -- 172.21.15.49:0/2155221488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d6c0c5680 msgr2=0x7f8d6c0c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d6be00640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d6c0c5680 0x7f8d6c0c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d6be00640 1 -- 172.21.15.49:0/2155221488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d6c0d0120 msgr2=0x7f8d6c0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d6be00640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d6c0d0120 0x7f8d6c0d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d6be00640 1 -- 172.21.15.49:0/2155221488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c0c97d0 con 0x7f8d500051c0 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.385+0000 7f8d70e00640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d6c0c5680 0x7f8d6c0c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.386+0000 7f8d6be00640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d6c15c1c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00e480 tx=0x7f8d5c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:54.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.386+0000 7f8d63e00640 1 -- 172.21.15.49:0/2155221488 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c0052e0 con 0x7f8d500051c0 2024-04-15T15:20:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.386+0000 7f8d63e00640 1 -- 172.21.15.49:0/2155221488 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5c018070 con 0x7f8d500051c0 2024-04-15T15:20:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.386+0000 7f8d71800640 1 -- 172.21.15.49:0/2155221488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c0c63f0 con 0x7f8d500051c0 2024-04-15T15:20:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.386+0000 7f8d63e00640 1 -- 172.21.15.49:0/2155221488 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c003830 con 0x7f8d500051c0 2024-04-15T15:20:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.386+0000 7f8d71800640 1 -- 172.21.15.49:0/2155221488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6c0c69b0 con 0x7f8d500051c0 2024-04-15T15:20:54.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.387+0000 7f8d71800640 1 -- 172.21.15.49:0/2155221488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d6c0755c0 con 0x7f8d500051c0 2024-04-15T15:20:54.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.388+0000 7f8d63e00640 1 -- 172.21.15.49:0/2155221488 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8d5c0039d0 con 0x7f8d500051c0 2024-04-15T15:20:54.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.389+0000 7f8d63e00640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8d4c06c9b0 0x7f8d4c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.389+0000 7f8d63e00640 1 -- 172.21.15.49:0/2155221488 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f8d5c08b5b0 con 0x7f8d500051c0 2024-04-15T15:20:54.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.390+0000 7f8d6b400640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8d4c06c9b0 0x7f8d4c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.390+0000 7f8d6b400640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8d4c06c9b0 0x7f8d4c06ee70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c145720 tx=0x7f8d58000ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:54.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.394+0000 7f8d63e00640 1 -- 172.21.15.49:0/2155221488 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d6c0c97d0 con 0x7f8d500051c0 2024-04-15T15:20:54.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.604+0000 7f8d71800640 1 -- 172.21.15.49:0/2155221488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_quiesce_splitauth"} v 0) -- 0x7f8d6c15c700 con 0x7f8d500051c0 2024-04-15T15:20:54.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.609+0000 7f8d63e00640 1 -- 172.21.15.49:0/2155221488 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_quiesce_splitauth"}]=0 v13) ==== 110+0+0 (secure 0 0 0) 0x7f8d5c059c20 con 0x7f8d500051c0 2024-04-15T15:20:54.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.612+0000 7f8d71800640 1 -- 172.21.15.49:0/2155221488 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8d4c06c9b0 msgr2=0x7f8d4c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.612+0000 7f8d71800640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8d4c06c9b0 0x7f8d4c06ee70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c145720 tx=0x7f8d58000ef0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.612+0000 7f8d71800640 1 -- 172.21.15.49:0/2155221488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 msgr2=0x7f8d6c15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.612+0000 7f8d71800640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d6c15c1c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00e480 tx=0x7f8d5c00e940 comp rx=0 tx=0).stop 2024-04-15T15:20:54.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.612+0000 7f8d71800640 1 -- 172.21.15.49:0/2155221488 shutdown_connections 2024-04-15T15:20:54.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.612+0000 7f8d71800640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8d4c06c9b0 0x7f8d4c06ee70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.612+0000 7f8d71800640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d6c0c5680 0x7f8d6c0c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.612+0000 7f8d71800640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d6c0d0120 0x7f8d6c0d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.612+0000 7f8d71800640 1 --2- 172.21.15.49:0/2155221488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d500051c0 0x7f8d6c15c1c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.612+0000 7f8d71800640 1 -- 172.21.15.49:0/2155221488 >> 172.21.15.49:0/2155221488 conn(0x7f8d6c06b930 msgr2=0x7f8d6c06f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:54.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.613+0000 7f8d71800640 1 -- 172.21.15.49:0/2155221488 shutdown_connections 2024-04-15T15:20:54.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.613+0000 7f8d71800640 1 -- 172.21.15.49:0/2155221488 wait complete. 2024-04-15T15:20:54.634 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-15T15:20:54.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.853+0000 7f55a3200640 1 Processor -- start 2024-04-15T15:20:54.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.853+0000 7f55a3200640 1 -- start start 2024-04-15T15:20:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.855+0000 7f55a3200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f55a40cebb0 0x7f55a4167590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.855+0000 7f55a3200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a40cd2d0 con 0x7f55a41461c0 2024-04-15T15:20:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.855+0000 7f55a3200640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a40cd5b0 con 0x7f55a40725f0 2024-04-15T15:20:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.855+0000 7f55a3200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f55a40cf1d0 con 0x7f55a40cebb0 2024-04-15T15:20:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.855+0000 7f55a1e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f55a41461c0 0x7f55a4071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42466/0 (socket says 172.21.15.49:42466) 2024-04-15T15:20:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.855+0000 7f55a1e00640 1 -- 172.21.15.49:0/1037811484 learned_addr learned my addr 172.21.15.49:0/1037811484 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:54.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1952528874 0 0) 0x7f55a40cd2d0 con 0x7f55a41461c0 2024-04-15T15:20:54.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a2800640 1 --2- 172.21.15.49:0/1037811484 >> v2:172.21.15.107:3300/0 conn(0x7f55a40cebb0 0x7f55a4167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5588003520 con 0x7f55a41461c0 2024-04-15T15:20:54.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1978154483 0 0) 0x7f5588003520 con 0x7f55a41461c0 2024-04-15T15:20:54.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 >> v2:172.21.15.107:3300/0 conn(0x7f55a40cebb0 msgr2=0x7f55a4167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:54.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 --2- 172.21.15.49:0/1037811484 >> v2:172.21.15.107:3300/0 conn(0x7f55a40cebb0 0x7f55a4167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 >> v1:172.21.15.107:6789/0 conn(0x7f55a40725f0 legacy=0x7f55a40ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a40cd890 con 0x7f55a41461c0 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f5594001670 con 0x7f55a41461c0 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 >> v1:172.21.15.49:6789/0 conn(0x7f55a41461c0 legacy=0x7f55a4071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 --2- 172.21.15.49:0/1037811484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55880039b0 0x7f5588003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.856+0000 7f55a0a00640 1 --2- 172.21.15.49:0/1037811484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5588004800 0x7f5588004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a0a00640 1 --2- 172.21.15.49:0/1037811484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55880051c0 0x7f558800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f558800c030 con 0x7f55880039b0 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f558800c1e0 con 0x7f5588004800 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f558800c3e0 con 0x7f55880051c0 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a1e00640 1 --2- 172.21.15.49:0/1037811484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55880039b0 0x7f5588003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a1400640 1 --2- 172.21.15.49:0/1037811484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5588004800 0x7f5588004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a1e00640 1 -- 172.21.15.49:0/1037811484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55880051c0 msgr2=0x7f558800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:54.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a1e00640 1 --2- 172.21.15.49:0/1037811484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55880051c0 0x7f558800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a1e00640 1 -- 172.21.15.49:0/1037811484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5588004800 msgr2=0x7f5588004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:54.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a1e00640 1 --2- 172.21.15.49:0/1037811484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5588004800 0x7f5588004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a1e00640 1 -- 172.21.15.49:0/1037811484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5588003520 con 0x7f55880039b0 2024-04-15T15:20:54.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.857+0000 7f55a2800640 1 -- 172.21.15.49:0/1037811484 reap_dead start 2024-04-15T15:20:54.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.858+0000 7f55a1e00640 1 --2- 172.21.15.49:0/1037811484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55880039b0 0x7f5588003dd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f559400c210 tx=0x7f5588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b2882ed74cc810 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:54.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.859+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594007d30 con 0x7f55880039b0 2024-04-15T15:20:54.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.859+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5594030070 con 0x7f55880039b0 2024-04-15T15:20:54.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.859+0000 7f55a0a00640 1 -- 172.21.15.49:0/1037811484 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594005d70 con 0x7f55880039b0 2024-04-15T15:20:54.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.859+0000 7f55a3200640 1 -- 172.21.15.49:0/1037811484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55880039b0 msgr2=0x7f5588003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.859+0000 7f55a3200640 1 --2- 172.21.15.49:0/1037811484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55880039b0 0x7f5588003dd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f559400c210 tx=0x7f5588005830 comp rx=0 tx=0).stop 2024-04-15T15:20:54.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.859+0000 7f55a3200640 1 -- 172.21.15.49:0/1037811484 shutdown_connections 2024-04-15T15:20:54.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.859+0000 7f55a3200640 1 --2- 172.21.15.49:0/1037811484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55880039b0 0x7f5588003dd0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.859+0000 7f55a3200640 1 -- 172.21.15.49:0/1037811484 >> 172.21.15.49:0/1037811484 conn(0x7f55a406b930 msgr2=0x7f55a406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:54.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.859+0000 7f55a3200640 1 -- 172.21.15.49:0/1037811484 shutdown_connections 2024-04-15T15:20:54.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.859+0000 7f55a3200640 1 -- 172.21.15.49:0/1037811484 wait complete. 2024-04-15T15:20:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.861+0000 7f55a3200640 1 Processor -- start 2024-04-15T15:20:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.861+0000 7f55a3200640 1 -- start start 2024-04-15T15:20:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.862+0000 7f55a3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55880039b0 0x7f55a415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.862+0000 7f55a3200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55a415c020 0x7f55a40a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.862+0000 7f55a3200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4156050 0x7f55a40b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.862+0000 7f55a3200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40ce4a0 con 0x7f55a4156050 2024-04-15T15:20:54.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.862+0000 7f55a3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40cd080 con 0x7f55880039b0 2024-04-15T15:20:54.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.862+0000 7f55a3200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f55a40ce620 con 0x7f55a415c020 2024-04-15T15:20:54.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a2800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4156050 0x7f55a40b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a2800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4156050 0x7f55a40b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43526/0 (socket says 172.21.15.49:43526) 2024-04-15T15:20:54.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a2800640 1 -- 172.21.15.49:0/455038124 learned_addr learned my addr 172.21.15.49:0/455038124 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:54.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a1400640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55a415c020 0x7f55a40a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a1e00640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55880039b0 0x7f55a415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a2800640 1 -- 172.21.15.49:0/455038124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55a415c020 msgr2=0x7f55a40a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a2800640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55a415c020 0x7f55a40a9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a2800640 1 -- 172.21.15.49:0/455038124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55880039b0 msgr2=0x7f55a415a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:54.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a2800640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55880039b0 0x7f55a415a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:54.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a2800640 1 -- 172.21.15.49:0/455038124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a4156500 con 0x7f55a4156050 2024-04-15T15:20:54.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.863+0000 7f55a1e00640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55880039b0 0x7f55a415a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:54.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.864+0000 7f55a2800640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4156050 0x7f55a40b1450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f55a4072e10 tx=0x7f559c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:54.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.864+0000 7f559a000640 1 -- 172.21.15.49:0/455038124 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c00bb70 con 0x7f55a4156050 2024-04-15T15:20:54.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.864+0000 7f559a000640 1 -- 172.21.15.49:0/455038124 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f559c01a070 con 0x7f55a4156050 2024-04-15T15:20:54.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.864+0000 7f559a000640 1 -- 172.21.15.49:0/455038124 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c014400 con 0x7f55a4156050 2024-04-15T15:20:54.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.864+0000 7f55a3200640 1 -- 172.21.15.49:0/455038124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a415af80 con 0x7f55a4156050 2024-04-15T15:20:54.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.864+0000 7f55a3200640 1 -- 172.21.15.49:0/455038124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a415b540 con 0x7f55a4156050 2024-04-15T15:20:54.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.866+0000 7f55a3200640 1 -- 172.21.15.49:0/455038124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a40755c0 con 0x7f55a4156050 2024-04-15T15:20:54.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.867+0000 7f559a000640 1 -- 172.21.15.49:0/455038124 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f559c005d10 con 0x7f55a4156050 2024-04-15T15:20:54.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.867+0000 7f559a000640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f558406c9b0 0x7f558406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:54.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.868+0000 7f559a000640 1 -- 172.21.15.49:0/455038124 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f559c08bbd0 con 0x7f55a4156050 2024-04-15T15:20:54.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.868+0000 7f55a1e00640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f558406c9b0 0x7f558406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:54.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.869+0000 7f55a1e00640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f558406c9b0 0x7f558406ee70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5594034000 tx=0x7f5594039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:54.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:54.872+0000 7f559a000640 1 -- 172.21.15.49:0/455038124 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f559c024020 con 0x7f55a4156050 2024-04-15T15:20:55.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.077+0000 7f55a3200640 1 -- 172.21.15.49:0/455038124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f55a4072e40 con 0x7f55a4156050 2024-04-15T15:20:55.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.082+0000 7f559a000640 1 -- 172.21.15.49:0/455038124 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) ==== 104+0+0 (secure 0 0 0) 0x7f559c05a1a0 con 0x7f55a4156050 2024-04-15T15:20:55.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.085+0000 7f55a3200640 1 -- 172.21.15.49:0/455038124 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f558406c9b0 msgr2=0x7f558406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.085+0000 7f55a3200640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f558406c9b0 0x7f558406ee70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5594034000 tx=0x7f5594039040 comp rx=0 tx=0).stop 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.085+0000 7f55a3200640 1 -- 172.21.15.49:0/455038124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4156050 msgr2=0x7f55a40b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.085+0000 7f55a3200640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4156050 0x7f55a40b1450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f55a4072e10 tx=0x7f559c00ef30 comp rx=0 tx=0).stop 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.086+0000 7f55a3200640 1 -- 172.21.15.49:0/455038124 shutdown_connections 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.086+0000 7f55a3200640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f558406c9b0 0x7f558406ee70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.086+0000 7f55a3200640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55a415c020 0x7f55a40a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.086+0000 7f55a3200640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4156050 0x7f55a40b1450 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.086+0000 7f55a3200640 1 --2- 172.21.15.49:0/455038124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55880039b0 0x7f55a415a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.086+0000 7f55a3200640 1 -- 172.21.15.49:0/455038124 >> 172.21.15.49:0/455038124 conn(0x7f55a406b930 msgr2=0x7f55a40b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.086+0000 7f55a3200640 1 -- 172.21.15.49:0/455038124 shutdown_connections 2024-04-15T15:20:55.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.086+0000 7f55a3200640 1 -- 172.21.15.49:0/455038124 wait complete. 2024-04-15T15:20:55.107 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds debug_mds 2024-04-15T15:20:55.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.332+0000 7fbe26000640 1 Processor -- start 2024-04-15T15:20:55.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.332+0000 7fbe26000640 1 -- start start 2024-04-15T15:20:55.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.333+0000 7fbe26000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbe200d1570 0x7fbe200cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.333+0000 7fbe26000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbe200ba130 0x7fbe200d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.333+0000 7fbe26000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe200cfc30 con 0x7fbe2015ed30 2024-04-15T15:20:55.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.333+0000 7fbe26000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fbe200d45e0 con 0x7fbe200ba130 2024-04-15T15:20:55.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.333+0000 7fbe26000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fbe200cff10 con 0x7fbe200d1570 2024-04-15T15:20:55.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.334+0000 7fbe25600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbe200ba130 0x7fbe200d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.334+0000 7fbe25600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbe200ba130 0x7fbe200d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43542/0 (socket says 172.21.15.49:43542) 2024-04-15T15:20:55.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.334+0000 7fbe25600640 1 -- 172.21.15.49:0/3411553881 learned_addr learned my addr 172.21.15.49:0/3411553881 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.334+0000 7fbe24c00640 1 --2- 172.21.15.49:0/3411553881 >> v2:172.21.15.107:3300/0 conn(0x7fbe200d1570 0x7fbe200cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.334+0000 7fbe25600640 1 -- 172.21.15.49:0/3411553881 >> v2:172.21.15.107:3300/0 conn(0x7fbe200d1570 msgr2=0x7fbe200cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.334+0000 7fbe25600640 1 --2- 172.21.15.49:0/3411553881 >> v2:172.21.15.107:3300/0 conn(0x7fbe200d1570 0x7fbe200cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.334+0000 7fbe25600640 1 -- 172.21.15.49:0/3411553881 >> v1:172.21.15.107:6789/0 conn(0x7fbe2015ed30 legacy=0x7fbe200d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.335+0000 7fbe25600640 1 -- 172.21.15.49:0/3411553881 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe200d35c0 con 0x7fbe200ba130 2024-04-15T15:20:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.335+0000 7fbe24c00640 1 --2- 172.21.15.49:0/3411553881 >> v2:172.21.15.107:3300/0 conn(0x7fbe200d1570 0x7fbe200cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:20:55.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.336+0000 7fbe25600640 1 --2- 172.21.15.49:0/3411553881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200ba130 0x7fbe200d40a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbe2006e040 tx=0x7fbe1802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4315b63180b7dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:55.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.336+0000 7fbe1f400640 1 -- 172.21.15.49:0/3411553881 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18034070 con 0x7fbe200ba130 2024-04-15T15:20:55.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.336+0000 7fbe1f400640 1 -- 172.21.15.49:0/3411553881 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe18030860 con 0x7fbe200ba130 2024-04-15T15:20:55.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.336+0000 7fbe1f400640 1 -- 172.21.15.49:0/3411553881 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18030c30 con 0x7fbe200ba130 2024-04-15T15:20:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.337+0000 7fbe26000640 1 -- 172.21.15.49:0/3411553881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200ba130 msgr2=0x7fbe200d40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.337+0000 7fbe26000640 1 --2- 172.21.15.49:0/3411553881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200ba130 0x7fbe200d40a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbe2006e040 tx=0x7fbe1802b9a0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.337+0000 7fbe26000640 1 -- 172.21.15.49:0/3411553881 shutdown_connections 2024-04-15T15:20:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.337+0000 7fbe26000640 1 --2- 172.21.15.49:0/3411553881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200ba130 0x7fbe200d40a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.337+0000 7fbe26000640 1 --2- 172.21.15.49:0/3411553881 >> v2:172.21.15.107:3300/0 conn(0x7fbe200d1570 0x7fbe200cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.337+0000 7fbe26000640 1 -- 172.21.15.49:0/3411553881 >> 172.21.15.49:0/3411553881 conn(0x7fbe20073240 msgr2=0x7fbe20073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.337+0000 7fbe26000640 1 -- 172.21.15.49:0/3411553881 shutdown_connections 2024-04-15T15:20:55.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.337+0000 7fbe26000640 1 -- 172.21.15.49:0/3411553881 wait complete. 2024-04-15T15:20:55.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.339+0000 7fbe26000640 1 Processor -- start 2024-04-15T15:20:55.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.339+0000 7fbe26000640 1 -- start start 2024-04-15T15:20:55.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe26000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe200ba130 0x7fbe201e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe26000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200d1570 0x7fbe201f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe26000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe2015ed30 0x7fbe20150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe26000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbe200ba5c0 con 0x7fbe200d1570 2024-04-15T15:20:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe26000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe200d3b80 con 0x7fbe200ba130 2024-04-15T15:20:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe26000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fbe200b78b0 con 0x7fbe2015ed30 2024-04-15T15:20:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe1fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200d1570 0x7fbe201f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe1fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200d1570 0x7fbe201f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43554/0 (socket says 172.21.15.49:43554) 2024-04-15T15:20:55.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe1fe00640 1 -- 172.21.15.49:0/2516647002 learned_addr learned my addr 172.21.15.49:0/2516647002 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:55.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe24c00640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe200ba130 0x7fbe201e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe25600640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe2015ed30 0x7fbe20150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe1fe00640 1 -- 172.21.15.49:0/2516647002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe2015ed30 msgr2=0x7fbe20150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:55.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe1fe00640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe2015ed30 0x7fbe20150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe1fe00640 1 -- 172.21.15.49:0/2516647002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe200ba130 msgr2=0x7fbe201e8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:55.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe1fe00640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe200ba130 0x7fbe201e8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.340+0000 7fbe1fe00640 1 -- 172.21.15.49:0/2516647002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe201f88a0 con 0x7fbe200d1570 2024-04-15T15:20:55.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.341+0000 7fbe1fe00640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200d1570 0x7fbe201f8360 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c440 tx=0x7fbe0800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:55.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.341+0000 7fbe1cc00640 1 -- 172.21.15.49:0/2516647002 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe080052d0 con 0x7fbe200d1570 2024-04-15T15:20:55.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.341+0000 7fbe1cc00640 1 -- 172.21.15.49:0/2516647002 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe08005470 con 0x7fbe200d1570 2024-04-15T15:20:55.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.341+0000 7fbe1cc00640 1 -- 172.21.15.49:0/2516647002 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08007260 con 0x7fbe200d1570 2024-04-15T15:20:55.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.341+0000 7fbe26000640 1 -- 172.21.15.49:0/2516647002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe201e91c0 con 0x7fbe200d1570 2024-04-15T15:20:55.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.341+0000 7fbe26000640 1 -- 172.21.15.49:0/2516647002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe201e9780 con 0x7fbe200d1570 2024-04-15T15:20:55.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.342+0000 7fbe26000640 1 -- 172.21.15.49:0/2516647002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe200755c0 con 0x7fbe200d1570 2024-04-15T15:20:55.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.343+0000 7fbe1cc00640 1 -- 172.21.15.49:0/2516647002 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fbe08007400 con 0x7fbe200d1570 2024-04-15T15:20:55.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.345+0000 7fbe1cc00640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbdfc06c980 0x7fbdfc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.345+0000 7fbe1cc00640 1 -- 172.21.15.49:0/2516647002 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fbe0808ae80 con 0x7fbe200d1570 2024-04-15T15:20:55.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.345+0000 7fbe24c00640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbdfc06c980 0x7fbdfc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.346+0000 7fbe24c00640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbdfc06c980 0x7fbdfc06ee40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbe10003fd0 tx=0x7fbe100039f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:55.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.349+0000 7fbe1cc00640 1 -- 172.21.15.49:0/2516647002 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe201f88a0 con 0x7fbe200d1570 2024-04-15T15:20:55.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.554+0000 7fbe26000640 1 -- 172.21.15.49:0/2516647002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "debug_mds"} v 0) -- 0x7fbe201e7550 con 0x7fbe200d1570 2024-04-15T15:20:55.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.559+0000 7fbe1cc00640 1 -- 172.21.15.49:0/2516647002 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "debug_mds"}]=0 v15) ==== 92+0+0 (secure 0 0 0) 0x7fbe08059460 con 0x7fbe200d1570 2024-04-15T15:20:55.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.562+0000 7fbe26000640 1 -- 172.21.15.49:0/2516647002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbdfc06c980 msgr2=0x7fbdfc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:55.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.562+0000 7fbe26000640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbdfc06c980 0x7fbdfc06ee40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbe10003fd0 tx=0x7fbe100039f0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.562+0000 7fbe26000640 1 -- 172.21.15.49:0/2516647002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200d1570 msgr2=0x7fbe201f8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.562+0000 7fbe26000640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200d1570 0x7fbe201f8360 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c440 tx=0x7fbe0800c900 comp rx=0 tx=0).stop 2024-04-15T15:20:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.563+0000 7fbe26000640 1 -- 172.21.15.49:0/2516647002 shutdown_connections 2024-04-15T15:20:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.563+0000 7fbe26000640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fbdfc06c980 0x7fbdfc06ee40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.563+0000 7fbe26000640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe2015ed30 0x7fbe20150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.563+0000 7fbe26000640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe200d1570 0x7fbe201f8360 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.563+0000 7fbe26000640 1 --2- 172.21.15.49:0/2516647002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe200ba130 0x7fbe201e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.563+0000 7fbe26000640 1 -- 172.21.15.49:0/2516647002 >> 172.21.15.49:0/2516647002 conn(0x7fbe20073240 msgr2=0x7fbe20160060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.563+0000 7fbe26000640 1 -- 172.21.15.49:0/2516647002 shutdown_connections 2024-04-15T15:20:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.563+0000 7fbe26000640 1 -- 172.21.15.49:0/2516647002 wait complete. 2024-04-15T15:20:55.585 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked' 2024-04-15T15:20:55.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.822+0000 7f321d600640 1 Processor -- start 2024-04-15T15:20:55.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.822+0000 7f321d600640 1 -- start start 2024-04-15T15:20:55.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.823+0000 7f321d600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32181461f0 0x7f32180ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.823+0000 7f321d600640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f32180b3fe0 0x7f321815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f321d600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32180b4be0 con 0x7f32180cedf0 2024-04-15T15:20:55.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f321d600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f32180a9640 con 0x7f32181461f0 2024-04-15T15:20:55.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f321d600640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f32180cf250 con 0x7f32180b3fe0 2024-04-15T15:20:55.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f3216a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32181461f0 0x7f32180ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f3216a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32181461f0 0x7f32180ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43556/0 (socket says 172.21.15.49:43556) 2024-04-15T15:20:55.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f3216a00640 1 -- 172.21.15.49:0/961482981 learned_addr learned my addr 172.21.15.49:0/961482981 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:55.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f3217400640 1 --2- 172.21.15.49:0/961482981 >> v2:172.21.15.107:3300/0 conn(0x7f32180b3fe0 0x7f321815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f3216a00640 1 -- 172.21.15.49:0/961482981 >> v2:172.21.15.107:3300/0 conn(0x7f32180b3fe0 msgr2=0x7f321815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:55.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f3216a00640 1 --2- 172.21.15.49:0/961482981 >> v2:172.21.15.107:3300/0 conn(0x7f32180b3fe0 0x7f321815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f3216a00640 1 -- 172.21.15.49:0/961482981 >> v1:172.21.15.107:6789/0 conn(0x7f32180cedf0 legacy=0x7f32180b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:55.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.824+0000 7f3216a00640 1 -- 172.21.15.49:0/961482981 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32180b4620 con 0x7f32181461f0 2024-04-15T15:20:55.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.826+0000 7f3216a00640 1 --2- 172.21.15.49:0/961482981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32181461f0 0x7f32180ce8b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f32180cf9a0 tx=0x7f320802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbc148a56e12fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:55.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.826+0000 7f3215600640 1 -- 172.21.15.49:0/961482981 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208035020 con 0x7f32181461f0 2024-04-15T15:20:55.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.826+0000 7f3215600640 1 -- 172.21.15.49:0/961482981 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3208031870 con 0x7f32181461f0 2024-04-15T15:20:55.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.826+0000 7f3215600640 1 -- 172.21.15.49:0/961482981 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208031c40 con 0x7f32181461f0 2024-04-15T15:20:55.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.827+0000 7f321d600640 1 -- 172.21.15.49:0/961482981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32181461f0 msgr2=0x7f32180ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:55.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.827+0000 7f321d600640 1 --2- 172.21.15.49:0/961482981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32181461f0 0x7f32180ce8b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f32180cf9a0 tx=0x7f320802b910 comp rx=0 tx=0).stop 2024-04-15T15:20:55.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.827+0000 7f321d600640 1 -- 172.21.15.49:0/961482981 shutdown_connections 2024-04-15T15:20:55.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.827+0000 7f321d600640 1 --2- 172.21.15.49:0/961482981 >> v2:172.21.15.107:3300/0 conn(0x7f32180b3fe0 0x7f321815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.827+0000 7f321d600640 1 --2- 172.21.15.49:0/961482981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32181461f0 0x7f32180ce8b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.827+0000 7f321d600640 1 -- 172.21.15.49:0/961482981 >> 172.21.15.49:0/961482981 conn(0x7f32180b27b0 msgr2=0x7f32180b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:55.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.827+0000 7f321d600640 1 -- 172.21.15.49:0/961482981 shutdown_connections 2024-04-15T15:20:55.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.827+0000 7f321d600640 1 -- 172.21.15.49:0/961482981 wait complete. 2024-04-15T15:20:55.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.829+0000 7f321d600640 1 Processor -- start 2024-04-15T15:20:55.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.829+0000 7f321d600640 1 -- start start 2024-04-15T15:20:55.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f321d600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180b3fe0 0x7f3218138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f321d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32180cedf0 0x7f32181e7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f321d600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32181461f0 0x7f32180aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f321d600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f321815d290 con 0x7f32180b3fe0 2024-04-15T15:20:55.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f321d600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f321815cfb0 con 0x7f32180cedf0 2024-04-15T15:20:55.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f321d600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f32181465d0 con 0x7f32181461f0 2024-04-15T15:20:55.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f3216a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180b3fe0 0x7f3218138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f3216a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180b3fe0 0x7f3218138cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43558/0 (socket says 172.21.15.49:43558) 2024-04-15T15:20:55.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f3216a00640 1 -- 172.21.15.49:0/2383522341 learned_addr learned my addr 172.21.15.49:0/2383522341 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:55.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f3216000640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32180cedf0 0x7f32181e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f3216a00640 1 -- 172.21.15.49:0/2383522341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32181461f0 msgr2=0x7f32180aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:55.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f3216a00640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32181461f0 0x7f32180aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f3216a00640 1 -- 172.21.15.49:0/2383522341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32180cedf0 msgr2=0x7f32181e7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:55.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.830+0000 7f3216a00640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32180cedf0 0x7f32181e7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:55.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.831+0000 7f3216a00640 1 -- 172.21.15.49:0/2383522341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32181e62d0 con 0x7f32180b3fe0 2024-04-15T15:20:55.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.831+0000 7f3216000640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32180cedf0 0x7f32181e7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:55.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.831+0000 7f3216a00640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180b3fe0 0x7f3218138cd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3208000c00 tx=0x7f32080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:55.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.831+0000 7f320ea00640 1 -- 172.21.15.49:0/2383522341 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208040070 con 0x7f32180b3fe0 2024-04-15T15:20:55.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.831+0000 7f320ea00640 1 -- 172.21.15.49:0/2383522341 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3208030d00 con 0x7f32180b3fe0 2024-04-15T15:20:55.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.831+0000 7f320ea00640 1 -- 172.21.15.49:0/2383522341 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208035040 con 0x7f32180b3fe0 2024-04-15T15:20:55.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.831+0000 7f321d600640 1 -- 172.21.15.49:0/2383522341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32181e65b0 con 0x7f32180b3fe0 2024-04-15T15:20:55.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.831+0000 7f321d600640 1 -- 172.21.15.49:0/2383522341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32181e6b70 con 0x7f32180b3fe0 2024-04-15T15:20:55.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.833+0000 7f321d600640 1 -- 172.21.15.49:0/2383522341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3218068c80 con 0x7f32180b3fe0 2024-04-15T15:20:55.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.833+0000 7f320ea00640 1 -- 172.21.15.49:0/2383522341 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f320804b060 con 0x7f32180b3fe0 2024-04-15T15:20:55.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.834+0000 7f320ea00640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f806c980 0x7f31f806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:55.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.835+0000 7f320ea00640 1 -- 172.21.15.49:0/2383522341 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f32080ad780 con 0x7f32180b3fe0 2024-04-15T15:20:55.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.835+0000 7f3216000640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f806c980 0x7f31f806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:55.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.835+0000 7f3216000640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f806c980 0x7f31f806ee40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3218145bc0 tx=0x7f3204009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:55.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:55.839+0000 7f320ea00640 1 -- 172.21.15.49:0/2383522341 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f320807bd60 con 0x7f32180b3fe0 2024-04-15T15:20:56.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.117+0000 7f321d600640 1 -- 172.21.15.49:0/2383522341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]} v 0) -- 0x7f321806feb0 con 0x7f32180b3fe0 2024-04-15T15:20:56.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.145+0000 7f320ea00640 1 -- 172.21.15.49:0/2383522341 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]}]=0 v138) ==== 137+0+0 (secure 0 0 0) 0x7f320807b660 con 0x7f32180b3fe0 2024-04-15T15:20:56.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.147+0000 7f321d600640 1 -- 172.21.15.49:0/2383522341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f806c980 msgr2=0x7f31f806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:56.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.147+0000 7f321d600640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f806c980 0x7f31f806ee40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3218145bc0 tx=0x7f3204009040 comp rx=0 tx=0).stop 2024-04-15T15:20:56.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.147+0000 7f321d600640 1 -- 172.21.15.49:0/2383522341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180b3fe0 msgr2=0x7f3218138cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:56.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.147+0000 7f321d600640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180b3fe0 0x7f3218138cd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3208000c00 tx=0x7f32080046c0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.148+0000 7f321d600640 1 -- 172.21.15.49:0/2383522341 shutdown_connections 2024-04-15T15:20:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.148+0000 7f321d600640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f31f806c980 0x7f31f806ee40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.148+0000 7f321d600640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32181461f0 0x7f32180aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.148+0000 7f321d600640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32180cedf0 0x7f32181e7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.148+0000 7f321d600640 1 --2- 172.21.15.49:0/2383522341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180b3fe0 0x7f3218138cd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.148+0000 7f321d600640 1 -- 172.21.15.49:0/2383522341 >> 172.21.15.49:0/2383522341 conn(0x7f32180b27b0 msgr2=0x7f321815de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.148+0000 7f321d600640 1 -- 172.21.15.49:0/2383522341 shutdown_connections 2024-04-15T15:20:56.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:56.148+0000 7f321d600640 1 -- 172.21.15.49:0/2383522341 wait complete. 2024-04-15T15:20:56.171 INFO:tasks.cephfs_test_runner:That a file with ceph.quiesce.block is not quiesced. ... FAIL 2024-04-15T15:20:56.171 INFO:tasks.cephfs_test_runner: 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner:FAIL: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner:That a file with ceph.quiesce.block is not quiesced. 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 541, in test_quiesce_blocked 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner: self._verify_quiesce(root=self.subvolume) 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 138, in _verify_quiesce 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner: return self._verify_quiesce_wrapped(rank, status, root, root_inode, ops, cache, splitauth) 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 272, in _verify_quiesce_wrapped 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner: self.assertFalse(lock_type.startswith("i")) 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner:AssertionError: True is not false 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner: 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner:Ran 1 test in 87.938s 2024-04-15T15:20:56.172 INFO:tasks.cephfs_test_runner: 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner: 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner:FAIL: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner:That a file with ceph.quiesce.block is not quiesced. 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 541, in test_quiesce_blocked 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner: self._verify_quiesce(root=self.subvolume) 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 138, in _verify_quiesce 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner: return self._verify_quiesce_wrapped(rank, status, root, root_inode, ops, cache, splitauth) 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 272, in _verify_quiesce_wrapped 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner: self.assertFalse(lock_type.startswith("i")) 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner:AssertionError: True is not false 2024-04-15T15:20:56.173 INFO:tasks.cephfs_test_runner: 2024-04-15T15:20:56.174 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:20:56.423 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=3c6838482d344462b31d935fb75c8cb5 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:20:56.424 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-15T15:20:56.434 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-15T15:20:56.447 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-15T15:20:56.447 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:20:56.447 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:56.511 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052756,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-15T15:20:56.511 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-15T15:20:56.511 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:56.568 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052756,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-15T15:20:56.569 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:20:56.569 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:56.625 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-15T15:20:56.625 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-15T15:20:56.625 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:20:56.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-15T15:20:56.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:56.679 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-04-15T15:20:56.741 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:20:56.742 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:20:56.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:56.743 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:20:56.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:56.855 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:20:56.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:56.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-04-15T15:20:57.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.082 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-04-15T15:20:57.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.129 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-04-15T15:20:57.270 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-04-15T15:20:57.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:20:57.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.274 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-15T15:20:57.332 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:20:57.333 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:20:57.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.333 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:20:57.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.431 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:20:57.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.516 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-04-15T15:20:57.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.634 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-15T15:20:57.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:20:57.677 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-04-15T15:20:57.808 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2024-04-15T15:20:57.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:20:57.812 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-15T15:20:57.822 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-15T15:20:57.822 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:20:58.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.062+0000 7f48b0a00640 1 Processor -- start 2024-04-15T15:20:58.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.062+0000 7f48b0a00640 1 -- start start 2024-04-15T15:20:58.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.064+0000 7f48b0a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f48ac15e7c0 0x7f48ac0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.064+0000 7f48b0a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48ac0b4be0 con 0x7f48ac0cefc0 2024-04-15T15:20:58.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.064+0000 7f48b0a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48ac0b4620 con 0x7f48ac1461f0 2024-04-15T15:20:58.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.064+0000 7f48b0a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f48ac0a9640 con 0x7f48ac15e7c0 2024-04-15T15:20:58.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.064+0000 7f48aa800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f48ac15e7c0 0x7f48ac0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.064+0000 7f48a1400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f48ac0cefc0 0x7f48ac0b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42472/0 (socket says 172.21.15.49:42472) 2024-04-15T15:20:58.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.064+0000 7f48a1400640 1 -- 172.21.15.49:0/252202388 learned_addr learned my addr 172.21.15.49:0/252202388 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:58.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.065+0000 7f48aa800640 1 -- 172.21.15.49:0/252202388 >> v1:172.21.15.107:6789/0 conn(0x7f48ac1461f0 legacy=0x7f48ac0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.065+0000 7f48aa800640 1 -- 172.21.15.49:0/252202388 >> v1:172.21.15.49:6789/0 conn(0x7f48ac0cefc0 legacy=0x7f48ac0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.065+0000 7f48aa800640 1 -- 172.21.15.49:0/252202388 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac0b4900 con 0x7f48ac15e7c0 2024-04-15T15:20:58.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.065+0000 7f48a9400640 1 -- 172.21.15.49:0/252202388 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3476907594 0 0) 0x7f48ac0b4be0 con 0x7f48ac0cefc0 2024-04-15T15:20:58.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.066+0000 7f48aa800640 1 --2- 172.21.15.49:0/252202388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac15e7c0 0x7f48ac0b3fe0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009d90 tx=0x7f48a402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a923f9464d1b81c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:58.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.066+0000 7f48a9400640 1 -- 172.21.15.49:0/252202388 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4039070 con 0x7f48ac15e7c0 2024-04-15T15:20:58.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.066+0000 7f48a9400640 1 -- 172.21.15.49:0/252202388 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a402cd00 con 0x7f48ac15e7c0 2024-04-15T15:20:58.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.066+0000 7f48a9400640 1 -- 172.21.15.49:0/252202388 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4034660 con 0x7f48ac15e7c0 2024-04-15T15:20:58.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.067+0000 7f48b0a00640 1 -- 172.21.15.49:0/252202388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac15e7c0 msgr2=0x7f48ac0b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.067+0000 7f48b0a00640 1 --2- 172.21.15.49:0/252202388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac15e7c0 0x7f48ac0b3fe0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009d90 tx=0x7f48a402bf50 comp rx=0 tx=0).stop 2024-04-15T15:20:58.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.067+0000 7f48b0a00640 1 -- 172.21.15.49:0/252202388 shutdown_connections 2024-04-15T15:20:58.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.067+0000 7f48b0a00640 1 --2- 172.21.15.49:0/252202388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac15e7c0 0x7f48ac0b3fe0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.067+0000 7f48b0a00640 1 -- 172.21.15.49:0/252202388 >> 172.21.15.49:0/252202388 conn(0x7f48ac0b27b0 msgr2=0x7f48ac0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:58.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.067+0000 7f48b0a00640 1 -- 172.21.15.49:0/252202388 shutdown_connections 2024-04-15T15:20:58.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.067+0000 7f48b0a00640 1 -- 172.21.15.49:0/252202388 wait complete. 2024-04-15T15:20:58.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.069+0000 7f48b0a00640 1 Processor -- start 2024-04-15T15:20:58.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.069+0000 7f48b0a00640 1 -- start start 2024-04-15T15:20:58.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.070+0000 7f48b0a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48ac0cefc0 0x7f48ac0ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.070+0000 7f48b0a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac1461f0 0x7f48ac1434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.070+0000 7f48b0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48ac15e7c0 0x7f48ac144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.070+0000 7f48b0a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac1465d0 con 0x7f48ac1461f0 2024-04-15T15:20:58.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.070+0000 7f48b0a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac15d290 con 0x7f48ac15e7c0 2024-04-15T15:20:58.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.070+0000 7f48b0a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f48ac167460 con 0x7f48ac0cefc0 2024-04-15T15:20:58.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.070+0000 7f48a1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac1461f0 0x7f48ac1434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.070+0000 7f48a1400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac1461f0 0x7f48ac1434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43576/0 (socket says 172.21.15.49:43576) 2024-04-15T15:20:58.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.070+0000 7f48a1400640 1 -- 172.21.15.49:0/2322268981 learned_addr learned my addr 172.21.15.49:0/2322268981 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:58.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.071+0000 7f48a1400640 1 -- 172.21.15.49:0/2322268981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48ac0cefc0 msgr2=0x7f48ac0ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.071+0000 7f48a9e00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48ac0cefc0 0x7f48ac0ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.071+0000 7f48aa800640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48ac15e7c0 0x7f48ac144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.071+0000 7f48a1400640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48ac0cefc0 0x7f48ac0ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.071+0000 7f48a1400640 1 -- 172.21.15.49:0/2322268981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48ac15e7c0 msgr2=0x7f48ac144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.071+0000 7f48a1400640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48ac15e7c0 0x7f48ac144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.071+0000 7f48a1400640 1 -- 172.21.15.49:0/2322268981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac142450 con 0x7f48ac1461f0 2024-04-15T15:20:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.071+0000 7f48a9e00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48ac0cefc0 0x7f48ac0ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.071+0000 7f48aa800640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48ac15e7c0 0x7f48ac144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.071+0000 7f48a1400640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac1461f0 0x7f48ac1434f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f489400c420 tx=0x7f489400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:58.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.072+0000 7f48a2a00640 1 -- 172.21.15.49:0/2322268981 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940074e0 con 0x7f48ac1461f0 2024-04-15T15:20:58.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.072+0000 7f48a2a00640 1 -- 172.21.15.49:0/2322268981 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48940047e0 con 0x7f48ac1461f0 2024-04-15T15:20:58.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.072+0000 7f48a2a00640 1 -- 172.21.15.49:0/2322268981 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894004c10 con 0x7f48ac1461f0 2024-04-15T15:20:58.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.072+0000 7f48b0a00640 1 -- 172.21.15.49:0/2322268981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac142730 con 0x7f48ac1461f0 2024-04-15T15:20:58.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.072+0000 7f48b0a00640 1 -- 172.21.15.49:0/2322268981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ac142cf0 con 0x7f48ac1461f0 2024-04-15T15:20:58.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.073+0000 7f48b0a00640 1 -- 172.21.15.49:0/2322268981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ac068c80 con 0x7f48ac1461f0 2024-04-15T15:20:58.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.074+0000 7f48a2a00640 1 -- 172.21.15.49:0/2322268981 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4894011070 con 0x7f48ac1461f0 2024-04-15T15:20:58.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.075+0000 7f48a2a00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488806c980 0x7f488806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.075+0000 7f48a2a00640 1 -- 172.21.15.49:0/2322268981 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f489408b5e0 con 0x7f48ac1461f0 2024-04-15T15:20:58.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.075+0000 7f48a9e00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488806c980 0x7f488806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.076+0000 7f48a9e00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488806c980 0x7f488806ee40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f489c000f60 tx=0x7f489c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:58.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.079+0000 7f48a2a00640 1 -- 172.21.15.49:0/2322268981 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f489400e280 con 0x7f48ac1461f0 2024-04-15T15:20:58.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.290+0000 7f48b0a00640 1 -- 172.21.15.49:0/2322268981 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f48ac06d1e0 con 0x7f488806c980 2024-04-15T15:20:58.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.291+0000 7f48a2a00640 1 -- 172.21.15.49:0/2322268981 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13496 (secure 0 0 0) 0x7f48ac06d1e0 con 0x7f488806c980 2024-04-15T15:20:58.293 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:58.294 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:20:58.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 -- 172.21.15.49:0/2322268981 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488806c980 msgr2=0x7f488806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488806c980 0x7f488806ee40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f489c000f60 tx=0x7f489c009040 comp rx=0 tx=0).stop 2024-04-15T15:20:58.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 -- 172.21.15.49:0/2322268981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac1461f0 msgr2=0x7f48ac1434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac1461f0 0x7f48ac1434f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f489400c420 tx=0x7f489400c8e0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 -- 172.21.15.49:0/2322268981 shutdown_connections 2024-04-15T15:20:58.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f488806c980 0x7f488806ee40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48ac15e7c0 0x7f48ac144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48ac1461f0 0x7f48ac1434f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 --2- 172.21.15.49:0/2322268981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48ac0cefc0 0x7f48ac0ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 -- 172.21.15.49:0/2322268981 >> 172.21.15.49:0/2322268981 conn(0x7f48ac0b27b0 msgr2=0x7f48ac0b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:58.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 -- 172.21.15.49:0/2322268981 shutdown_connections 2024-04-15T15:20:58.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.294+0000 7f48b0a00640 1 -- 172.21.15.49:0/2322268981 wait complete. 2024-04-15T15:20:58.315 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-04-15T15:20:56.815666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237456,"kb_used_data":4656,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737264,"statfs":{"total":773094113280,"available":772850958336,"internally_reserved":0,"allocated":4767744,"data_stored":2683154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-74746,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-132,"num_write":-107,"num_write_kb":-119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002185"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":200,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-15T15:20:51.120039+0000","last_change":"2024-04-15T15:18:41.162887+0000","last_active":"2024-04-15T15:20:51.120039+0000","last_peered":"2024-04-15T15:20:51.120039+0000","last_clean":"2024-04-15T15:20:51.120039+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:20:51.120039+0000","last_undegraded":"2024-04-15T15:20:51.120039+0000","last_fullsized":"2024-04-15T15:20:51.120039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_clean_scrub_stamp":"2024-04-15T15:18:39.644354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:32:00.047414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":757288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":166934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":516096,"data_stored":249810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":421888,"data_stored":166900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":249943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1015808,"data_stored":757331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:20:58.315 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:20:58.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.541+0000 7fdfaf200640 1 Processor -- start 2024-04-15T15:20:58.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.541+0000 7fdfaf200640 1 -- start start 2024-04-15T15:20:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.543+0000 7fdfaf200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015b860 0x7fdfb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.543+0000 7fdfaf200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdfb006f320 0x7fdfb00b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.543+0000 7fdfaf200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb00cdfb0 con 0x7fdfb015b480 2024-04-15T15:20:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.543+0000 7fdfaf200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb015f060 con 0x7fdfb006f320 2024-04-15T15:20:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.543+0000 7fdfaf200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb015fec0 con 0x7fdfb015b860 2024-04-15T15:20:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.543+0000 7fdfade00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdfb015b480 0x7fdfb00cd8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42474/0 (socket says 172.21.15.49:42474) 2024-04-15T15:20:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.543+0000 7fdfade00640 1 -- 172.21.15.49:0/4076422231 learned_addr learned my addr 172.21.15.49:0/4076422231 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:58.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfae800640 1 --2- 172.21.15.49:0/4076422231 >> v2:172.21.15.107:3300/0 conn(0x7fdfb006f320 0x7fdfb00b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1613605383 0 0) 0x7fdfb00cdfb0 con 0x7fdfb015b480 2024-04-15T15:20:58.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfad400640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015b860 0x7fdfb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf94003520 con 0x7fdfb015b480 2024-04-15T15:20:58.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1145712444 0 0) 0x7fdf94003520 con 0x7fdfb015b480 2024-04-15T15:20:58.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015b860 msgr2=0x7fdfb006ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:58.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015b860 0x7fdfb006ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 >> v2:172.21.15.107:3300/0 conn(0x7fdfb006f320 msgr2=0x7fdfb00b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:58.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 --2- 172.21.15.49:0/4076422231 >> v2:172.21.15.107:3300/0 conn(0x7fdfb006f320 0x7fdfb00b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb015f2b0 con 0x7fdfb015b480 2024-04-15T15:20:58.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7fdfa0001670 con 0x7fdfb015b480 2024-04-15T15:20:58.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 >> v1:172.21.15.49:6789/0 conn(0x7fdfb015b480 legacy=0x7fdfb00cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015b860 0x7fdf940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf94003f30 0x7fdf94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf94004d40 0x7fdf940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb015f060 con 0x7fdf94004d40 2024-04-15T15:20:58.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb015fec0 con 0x7fdf94003f30 2024-04-15T15:20:58.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.544+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9400b9c0 con 0x7fdfb015b860 2024-04-15T15:20:58.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.545+0000 7fdfae800640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf94004d40 0x7fdf940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.545+0000 7fdfade00640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015b860 0x7fdf940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.545+0000 7fdfae800640 1 -- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015b860 msgr2=0x7fdf940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:58.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.545+0000 7fdfae800640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015b860 0x7fdf940039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.545+0000 7fdfae800640 1 -- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf94003f30 msgr2=0x7fdf94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:58.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.545+0000 7fdfad400640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf94003f30 0x7fdf94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.545+0000 7fdfae800640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf94003f30 0x7fdf94004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.545+0000 7fdfae800640 1 -- 172.21.15.49:0/4076422231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf94003520 con 0x7fdf94004d40 2024-04-15T15:20:58.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.545+0000 7fdfade00640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015b860 0x7fdf940039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.546+0000 7fdfae800640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf94004d40 0x7fdf940051c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009fc0 tx=0x7fdf94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548cbe26f5fe4999 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.547+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8039070 con 0x7fdf94004d40 2024-04-15T15:20:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.547+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa802cc40 con 0x7fdf94004d40 2024-04-15T15:20:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.547+0000 7fdfaca00640 1 -- 172.21.15.49:0/4076422231 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8009c10 con 0x7fdf94004d40 2024-04-15T15:20:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.547+0000 7fdfaf200640 1 -- 172.21.15.49:0/4076422231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf94004d40 msgr2=0x7fdf940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.547+0000 7fdfaf200640 1 --2- 172.21.15.49:0/4076422231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf94004d40 0x7fdf940051c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009fc0 tx=0x7fdf94005830 comp rx=0 tx=0).stop 2024-04-15T15:20:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.547+0000 7fdfae800640 1 -- 172.21.15.49:0/4076422231 reap_dead start 2024-04-15T15:20:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.547+0000 7fdfaf200640 1 -- 172.21.15.49:0/4076422231 shutdown_connections 2024-04-15T15:20:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.547+0000 7fdfaf200640 1 -- 172.21.15.49:0/4076422231 >> 172.21.15.49:0/4076422231 conn(0x7fdfb006b930 msgr2=0x7fdfb006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.547+0000 7fdfaf200640 1 -- 172.21.15.49:0/4076422231 shutdown_connections 2024-04-15T15:20:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.548+0000 7fdfaf200640 1 -- 172.21.15.49:0/4076422231 wait complete. 2024-04-15T15:20:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.549+0000 7fdfaf200640 1 Processor -- start 2024-04-15T15:20:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.549+0000 7fdfaf200640 1 -- start start 2024-04-15T15:20:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.550+0000 7fdfaf200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb0142370 0x7fdfb0145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.550+0000 7fdfaf200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015cd80 0x7fdfb015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.550+0000 7fdfaf200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb01f5960 0x7fdfb01f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.550+0000 7fdfaf200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb015fb50 con 0x7fdfb0142370 2024-04-15T15:20:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.550+0000 7fdfaf200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb015f870 con 0x7fdfb01f5960 2024-04-15T15:20:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.550+0000 7fdfaf200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb00cd550 con 0x7fdfb015cd80 2024-04-15T15:20:58.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.551+0000 7fdfade00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb0142370 0x7fdfb0145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.551+0000 7fdfade00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb0142370 0x7fdfb0145670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43588/0 (socket says 172.21.15.49:43588) 2024-04-15T15:20:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.551+0000 7fdfade00640 1 -- 172.21.15.49:0/360919023 learned_addr learned my addr 172.21.15.49:0/360919023 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.551+0000 7fdfad400640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015cd80 0x7fdfb015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.551+0000 7fdfade00640 1 -- 172.21.15.49:0/360919023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015cd80 msgr2=0x7fdfb015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.551+0000 7fdfae800640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb01f5960 0x7fdfb01f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.551+0000 7fdfade00640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015cd80 0x7fdfb015d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.551+0000 7fdfade00640 1 -- 172.21.15.49:0/360919023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb01f5960 msgr2=0x7fdfb01f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.551+0000 7fdfade00640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb01f5960 0x7fdfb01f5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.551+0000 7fdfade00640 1 -- 172.21.15.49:0/360919023 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb0145bb0 con 0x7fdfb0142370 2024-04-15T15:20:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.552+0000 7fdfade00640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb0142370 0x7fdfb0145670 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000db10 tx=0x7fdfa0007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.552+0000 7fdfa6000640 1 -- 172.21.15.49:0/360919023 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0019070 con 0x7fdfb0142370 2024-04-15T15:20:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.552+0000 7fdfa6000640 1 -- 172.21.15.49:0/360919023 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa0004cb0 con 0x7fdfb0142370 2024-04-15T15:20:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.552+0000 7fdfa6000640 1 -- 172.21.15.49:0/360919023 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0004e60 con 0x7fdfb0142370 2024-04-15T15:20:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.552+0000 7fdfaf200640 1 -- 172.21.15.49:0/360919023 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb01e51c0 con 0x7fdfb0142370 2024-04-15T15:20:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.552+0000 7fdfaf200640 1 -- 172.21.15.49:0/360919023 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb01e5780 con 0x7fdfb0142370 2024-04-15T15:20:58.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.553+0000 7fdfaf200640 1 -- 172.21.15.49:0/360919023 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb006fc50 con 0x7fdfb0142370 2024-04-15T15:20:58.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.554+0000 7fdfa6000640 1 -- 172.21.15.49:0/360919023 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fdfa00072d0 con 0x7fdfb0142370 2024-04-15T15:20:58.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.555+0000 7fdfa6000640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdf9006c980 0x7fdf9006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:58.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.555+0000 7fdfa6000640 1 -- 172.21.15.49:0/360919023 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fdfa008b480 con 0x7fdfb0142370 2024-04-15T15:20:58.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.555+0000 7fdfad400640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdf9006c980 0x7fdf9006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:58.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.556+0000 7fdfad400640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdf9006c980 0x7fdf9006ee40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdfb015da80 tx=0x7fdf9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:58.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.560+0000 7fdfa6000640 1 -- 172.21.15.49:0/360919023 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdfb0145bb0 con 0x7fdfb0142370 2024-04-15T15:20:58.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.762+0000 7fdfaf200640 1 -- 172.21.15.49:0/360919023 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfb00755c0 con 0x7fdf9006c980 2024-04-15T15:20:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.764+0000 7fdfa6000640 1 -- 172.21.15.49:0/360919023 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13496 (secure 0 0 0) 0x7fdfb00755c0 con 0x7fdf9006c980 2024-04-15T15:20:58.766 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:20:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.766+0000 7fdfaf200640 1 -- 172.21.15.49:0/360919023 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdf9006c980 msgr2=0x7fdf9006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.766+0000 7fdfaf200640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdf9006c980 0x7fdf9006ee40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdfb015da80 tx=0x7fdf9c008040 comp rx=0 tx=0).stop 2024-04-15T15:20:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.766+0000 7fdfaf200640 1 -- 172.21.15.49:0/360919023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb0142370 msgr2=0x7fdfb0145670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.766+0000 7fdfaf200640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb0142370 0x7fdfb0145670 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000db10 tx=0x7fdfa0007e30 comp rx=0 tx=0).stop 2024-04-15T15:20:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.766+0000 7fdfaf200640 1 -- 172.21.15.49:0/360919023 shutdown_connections 2024-04-15T15:20:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.766+0000 7fdfaf200640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fdf9006c980 0x7fdf9006ee40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.766+0000 7fdfaf200640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdfb015cd80 0x7fdfb015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.766+0000 7fdfaf200640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb01f5960 0x7fdfb01f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.766+0000 7fdfaf200640 1 --2- 172.21.15.49:0/360919023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb0142370 0x7fdfb0145670 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.767+0000 7fdfaf200640 1 -- 172.21.15.49:0/360919023 >> 172.21.15.49:0/360919023 conn(0x7fdfb006b930 msgr2=0x7fdfb006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.767+0000 7fdfaf200640 1 -- 172.21.15.49:0/360919023 shutdown_connections 2024-04-15T15:20:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:58.767+0000 7fdfaf200640 1 -- 172.21.15.49:0/360919023 wait complete. 2024-04-15T15:20:58.789 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-04-15T15:20:56.815666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237456,"kb_used_data":4656,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737264,"statfs":{"total":773094113280,"available":772850958336,"internally_reserved":0,"allocated":4767744,"data_stored":2683154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-74746,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-132,"num_write":-107,"num_write_kb":-119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002185"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":200,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-15T15:20:51.120039+0000","last_change":"2024-04-15T15:18:41.162887+0000","last_active":"2024-04-15T15:20:51.120039+0000","last_peered":"2024-04-15T15:20:51.120039+0000","last_clean":"2024-04-15T15:20:51.120039+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:20:51.120039+0000","last_undegraded":"2024-04-15T15:20:51.120039+0000","last_fullsized":"2024-04-15T15:20:51.120039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_clean_scrub_stamp":"2024-04-15T15:18:39.644354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:32:00.047414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":757288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":166934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":516096,"data_stored":249810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":421888,"data_stored":166900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":249943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1015808,"data_stored":757331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:20:58.790 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-15T15:20:58.790 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:20:59.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.025+0000 7ff893200640 1 Processor -- start 2024-04-15T15:20:59.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.025+0000 7ff893200640 1 -- start start 2024-04-15T15:20:59.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.026+0000 7ff893200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff894071d10 0x7ff894070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.026+0000 7ff893200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff8940cde40 0x7ff8940ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff893200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8940cd0c0 con 0x7ff89409aa40 2024-04-15T15:20:59.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff893200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff8940710f0 con 0x7ff894071d10 2024-04-15T15:20:59.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff893200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7ff8940cd360 con 0x7ff8940cde40 2024-04-15T15:20:59.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff891400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff894071d10 0x7ff894070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff891e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff89409aa40 0x7ff89409ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42488/0 (socket says 172.21.15.49:42488) 2024-04-15T15:20:59.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff891e00640 1 -- 172.21.15.49:0/2355350993 learned_addr learned my addr 172.21.15.49:0/2355350993 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:20:59.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff892800640 1 --2- 172.21.15.49:0/2355350993 >> v2:172.21.15.107:3300/0 conn(0x7ff8940cde40 0x7ff8940ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff891400640 1 -- 172.21.15.49:0/2355350993 >> v2:172.21.15.107:3300/0 conn(0x7ff8940cde40 msgr2=0x7ff8940ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:59.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff891400640 1 --2- 172.21.15.49:0/2355350993 >> v2:172.21.15.107:3300/0 conn(0x7ff8940cde40 0x7ff8940ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff891400640 1 -- 172.21.15.49:0/2355350993 >> v1:172.21.15.49:6789/0 conn(0x7ff89409aa40 legacy=0x7ff89409ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.027+0000 7ff891400640 1 -- 172.21.15.49:0/2355350993 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8940cd640 con 0x7ff894071d10 2024-04-15T15:20:59.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.029+0000 7ff891400640 1 --2- 172.21.15.49:0/2355350993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 0x7ff894070bb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff894073930 tx=0x7ff88002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1804dae56da37bd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:59.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.029+0000 7ff890a00640 1 -- 172.21.15.49:0/2355350993 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880035020 con 0x7ff894071d10 2024-04-15T15:20:59.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.029+0000 7ff890a00640 1 -- 172.21.15.49:0/2355350993 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff880031830 con 0x7ff894071d10 2024-04-15T15:20:59.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.029+0000 7ff890a00640 1 -- 172.21.15.49:0/2355350993 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880031c00 con 0x7ff894071d10 2024-04-15T15:20:59.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.030+0000 7ff893200640 1 -- 172.21.15.49:0/2355350993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 msgr2=0x7ff894070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.030+0000 7ff893200640 1 --2- 172.21.15.49:0/2355350993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 0x7ff894070bb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff894073930 tx=0x7ff88002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.030+0000 7ff893200640 1 -- 172.21.15.49:0/2355350993 shutdown_connections 2024-04-15T15:20:59.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.030+0000 7ff893200640 1 --2- 172.21.15.49:0/2355350993 >> v2:172.21.15.107:3300/0 conn(0x7ff8940cde40 0x7ff8940ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.030+0000 7ff893200640 1 --2- 172.21.15.49:0/2355350993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 0x7ff894070bb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.030+0000 7ff893200640 1 -- 172.21.15.49:0/2355350993 >> 172.21.15.49:0/2355350993 conn(0x7ff8940d2540 msgr2=0x7ff8940d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:59.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.030+0000 7ff893200640 1 -- 172.21.15.49:0/2355350993 shutdown_connections 2024-04-15T15:20:59.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.030+0000 7ff893200640 1 -- 172.21.15.49:0/2355350993 wait complete. 2024-04-15T15:20:59.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.032+0000 7ff893200640 1 Processor -- start 2024-04-15T15:20:59.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.032+0000 7ff893200640 1 -- start start 2024-04-15T15:20:59.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff893200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 0x7ff894155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff893200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff89409aa40 0x7ff89409e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff893200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8940cde40 0x7ff89409eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff893200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff894071510 con 0x7ff894071d10 2024-04-15T15:20:59.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff893200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8940721f0 con 0x7ff8940cde40 2024-04-15T15:20:59.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff893200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff8940af3f0 con 0x7ff89409aa40 2024-04-15T15:20:59.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff891e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 0x7ff894155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff891e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 0x7ff894155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43606/0 (socket says 172.21.15.49:43606) 2024-04-15T15:20:59.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff891e00640 1 -- 172.21.15.49:0/3900216666 learned_addr learned my addr 172.21.15.49:0/3900216666 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:59.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff892800640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8940cde40 0x7ff89409eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff891e00640 1 -- 172.21.15.49:0/3900216666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff89409aa40 msgr2=0x7ff89409e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.033+0000 7ff891e00640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff89409aa40 0x7ff89409e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.034+0000 7ff891e00640 1 -- 172.21.15.49:0/3900216666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8940cde40 msgr2=0x7ff89409eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.034+0000 7ff891e00640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8940cde40 0x7ff89409eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.034+0000 7ff891e00640 1 -- 172.21.15.49:0/3900216666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89406d7c0 con 0x7ff894071d10 2024-04-15T15:20:59.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.034+0000 7ff891e00640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 0x7ff894155830 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff88400c400 tx=0x7ff88400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:59.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.034+0000 7ff88a000640 1 -- 172.21.15.49:0/3900216666 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884017070 con 0x7ff894071d10 2024-04-15T15:20:59.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.034+0000 7ff88a000640 1 -- 172.21.15.49:0/3900216666 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff884005ce0 con 0x7ff894071d10 2024-04-15T15:20:59.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.034+0000 7ff88a000640 1 -- 172.21.15.49:0/3900216666 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8840072f0 con 0x7ff894071d10 2024-04-15T15:20:59.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.034+0000 7ff893200640 1 -- 172.21.15.49:0/3900216666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89406daa0 con 0x7ff894071d10 2024-04-15T15:20:59.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.034+0000 7ff893200640 1 -- 172.21.15.49:0/3900216666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89406e060 con 0x7ff894071d10 2024-04-15T15:20:59.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.036+0000 7ff893200640 1 -- 172.21.15.49:0/3900216666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8940755c0 con 0x7ff894071d10 2024-04-15T15:20:59.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.036+0000 7ff88a000640 1 -- 172.21.15.49:0/3900216666 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff88401a070 con 0x7ff894071d10 2024-04-15T15:20:59.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.037+0000 7ff88a000640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff87406c980 0x7ff87406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.037+0000 7ff88a000640 1 -- 172.21.15.49:0/3900216666 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7ff88408ac70 con 0x7ff894071d10 2024-04-15T15:20:59.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.038+0000 7ff891400640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff87406c980 0x7ff87406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.038+0000 7ff891400640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff87406c980 0x7ff87406ee40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff880036000 tx=0x7ff880030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:59.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.042+0000 7ff88a000640 1 -- 172.21.15.49:0/3900216666 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8840593e0 con 0x7ff894071d10 2024-04-15T15:20:59.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.244+0000 7ff893200640 1 -- 172.21.15.49:0/3900216666 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff894155d70 con 0x7ff87406c980 2024-04-15T15:20:59.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.246+0000 7ff88a000640 1 -- 172.21.15.49:0/3900216666 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13490 (secure 0 0 0) 0x7ff894155d70 con 0x7ff87406c980 2024-04-15T15:20:59.248 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:20:59.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 -- 172.21.15.49:0/3900216666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff87406c980 msgr2=0x7ff87406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff87406c980 0x7ff87406ee40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff880036000 tx=0x7ff880030000 comp rx=0 tx=0).stop 2024-04-15T15:20:59.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 -- 172.21.15.49:0/3900216666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 msgr2=0x7ff894155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 0x7ff894155830 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff88400c400 tx=0x7ff88400c8c0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 -- 172.21.15.49:0/3900216666 shutdown_connections 2024-04-15T15:20:59.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7ff87406c980 0x7ff87406ee40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff89409aa40 0x7ff89409e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8940cde40 0x7ff89409eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 --2- 172.21.15.49:0/3900216666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff894071d10 0x7ff894155830 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 -- 172.21.15.49:0/3900216666 >> 172.21.15.49:0/3900216666 conn(0x7ff8940d2540 msgr2=0x7ff8940d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:59.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 -- 172.21.15.49:0/3900216666 shutdown_connections 2024-04-15T15:20:59.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.249+0000 7ff893200640 1 -- 172.21.15.49:0/3900216666 wait complete. 2024-04-15T15:20:59.269 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-15T15:20:58.816028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237480,"kb_used_data":4680,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737240,"statfs":{"total":773094113280,"available":772850933760,"internally_reserved":0,"allocated":4792320,"data_stored":2609296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-74746,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-132,"num_write":-107,"num_write_kb":-119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.701515"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":204,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-15T15:20:54.143676+0000","last_change":"2024-04-15T15:18:41.162887+0000","last_active":"2024-04-15T15:20:54.143676+0000","last_peered":"2024-04-15T15:20:54.143676+0000","last_clean":"2024-04-15T15:20:54.143676+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:20:54.143676+0000","last_undegraded":"2024-04-15T15:20:54.143676+0000","last_fullsized":"2024-04-15T15:20:54.143676+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_clean_scrub_stamp":"2024-04-15T15:18:39.644354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:32:00.047414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":768938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542175,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":768938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:20:59.270 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:20:59.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.499+0000 7fb71d400640 1 Processor -- start 2024-04-15T15:20:59.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.499+0000 7fb71d400640 1 -- start start 2024-04-15T15:20:59.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.500+0000 7fb71d400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71815b480 0x7fb7180cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.500+0000 7fb71d400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb71815b860 0x7fb7180b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.500+0000 7fb71d400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71815f2b0 con 0x7fb7180cdde0 2024-04-15T15:20:59.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.500+0000 7fb71d400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb71815f060 con 0x7fb71815b860 2024-04-15T15:20:59.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.500+0000 7fb71d400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb71815fec0 con 0x7fb71815b480 2024-04-15T15:20:59.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.501+0000 7fb71ca00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb71815b860 0x7fb7180b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.501+0000 7fb71ca00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb71815b860 0x7fb7180b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43618/0 (socket says 172.21.15.49:43618) 2024-04-15T15:20:59.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.501+0000 7fb71ca00640 1 -- 172.21.15.49:0/183520269 learned_addr learned my addr 172.21.15.49:0/183520269 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:59.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.501+0000 7fb717e00640 1 --2- 172.21.15.49:0/183520269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71815b480 0x7fb7180cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.501+0000 7fb71ca00640 1 -- 172.21.15.49:0/183520269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71815b480 msgr2=0x7fb7180cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:59.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.501+0000 7fb71ca00640 1 --2- 172.21.15.49:0/183520269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71815b480 0x7fb7180cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.501+0000 7fb71ca00640 1 -- 172.21.15.49:0/183520269 >> v1:172.21.15.107:6789/0 conn(0x7fb7180cdde0 legacy=0x7fb71806ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:20:59.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.501+0000 7fb71ca00640 1 -- 172.21.15.49:0/183520269 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71815f590 con 0x7fb71815b860 2024-04-15T15:20:59.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.503+0000 7fb71ca00640 1 --2- 172.21.15.49:0/183520269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71815b860 0x7fb7180b3fc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb71815f1e0 tx=0x7fb71002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c414ce135380cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:59.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.503+0000 7fb716a00640 1 -- 172.21.15.49:0/183520269 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71002ec60 con 0x7fb71815b860 2024-04-15T15:20:59.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.503+0000 7fb716a00640 1 -- 172.21.15.49:0/183520269 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71002ee00 con 0x7fb71815b860 2024-04-15T15:20:59.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.503+0000 7fb716a00640 1 -- 172.21.15.49:0/183520269 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710036610 con 0x7fb71815b860 2024-04-15T15:20:59.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.504+0000 7fb71d400640 1 -- 172.21.15.49:0/183520269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71815b860 msgr2=0x7fb7180b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.504+0000 7fb71d400640 1 --2- 172.21.15.49:0/183520269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71815b860 0x7fb7180b3fc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb71815f1e0 tx=0x7fb71002e0c0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.504+0000 7fb71d400640 1 -- 172.21.15.49:0/183520269 shutdown_connections 2024-04-15T15:20:59.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.504+0000 7fb71d400640 1 --2- 172.21.15.49:0/183520269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb71815b860 0x7fb7180b3fc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.504+0000 7fb71d400640 1 --2- 172.21.15.49:0/183520269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71815b480 0x7fb7180cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.504+0000 7fb71d400640 1 -- 172.21.15.49:0/183520269 >> 172.21.15.49:0/183520269 conn(0x7fb71806b930 msgr2=0x7fb71806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:59.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.504+0000 7fb71d400640 1 -- 172.21.15.49:0/183520269 shutdown_connections 2024-04-15T15:20:59.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.504+0000 7fb71d400640 1 -- 172.21.15.49:0/183520269 wait complete. 2024-04-15T15:20:59.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.505+0000 7fb71d400640 1 Processor -- start 2024-04-15T15:20:59.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.506+0000 7fb71d400640 1 -- start start 2024-04-15T15:20:59.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.506+0000 7fb71d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7180cdde0 0x7fb7181e6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb71d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71815b480 0x7fb7181f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb71d400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71815b860 0x7fb718146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb71d400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb7180cd550 con 0x7fb7180cdde0 2024-04-15T15:20:59.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb71d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb71815fb50 con 0x7fb71815b480 2024-04-15T15:20:59.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb71d400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb71806f4f0 con 0x7fb71815b860 2024-04-15T15:20:59.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb717e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7180cdde0 0x7fb7181e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb717e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7180cdde0 0x7fb7181e6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43626/0 (socket says 172.21.15.49:43626) 2024-04-15T15:20:59.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb717e00640 1 -- 172.21.15.49:0/2596388386 learned_addr learned my addr 172.21.15.49:0/2596388386 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:20:59.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb71ca00640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71815b860 0x7fb718146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb717400640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71815b480 0x7fb7181f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb717e00640 1 -- 172.21.15.49:0/2596388386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71815b860 msgr2=0x7fb718146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb717e00640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71815b860 0x7fb718146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb717e00640 1 -- 172.21.15.49:0/2596388386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71815b480 msgr2=0x7fb7181f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.507+0000 7fb717e00640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71815b480 0x7fb7181f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.508+0000 7fb717e00640 1 -- 172.21.15.49:0/2596388386 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7181f5100 con 0x7fb7180cdde0 2024-04-15T15:20:59.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.508+0000 7fb717400640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71815b480 0x7fb7181f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:20:59.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.508+0000 7fb717e00640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7180cdde0 0x7fb7181e6430 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb70800d800 tx=0x7fb70800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:59.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.508+0000 7fb70fe00640 1 -- 172.21.15.49:0/2596388386 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70800b6c0 con 0x7fb7180cdde0 2024-04-15T15:20:59.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.508+0000 7fb70fe00640 1 -- 172.21.15.49:0/2596388386 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb708019070 con 0x7fb7180cdde0 2024-04-15T15:20:59.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.508+0000 7fb70fe00640 1 -- 172.21.15.49:0/2596388386 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70800ba80 con 0x7fb7180cdde0 2024-04-15T15:20:59.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.508+0000 7fb71d400640 1 -- 172.21.15.49:0/2596388386 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7181e5390 con 0x7fb7180cdde0 2024-04-15T15:20:59.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.508+0000 7fb71d400640 1 -- 172.21.15.49:0/2596388386 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7181e5950 con 0x7fb7180cdde0 2024-04-15T15:20:59.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.509+0000 7fb71d400640 1 -- 172.21.15.49:0/2596388386 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7180755c0 con 0x7fb7180cdde0 2024-04-15T15:20:59.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.510+0000 7fb70fe00640 1 -- 172.21.15.49:0/2596388386 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb70800bc20 con 0x7fb7180cdde0 2024-04-15T15:20:59.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.511+0000 7fb70fe00640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6f806c980 0x7fb6f806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:20:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.511+0000 7fb70fe00640 1 -- 172.21.15.49:0/2596388386 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fb70808bee0 con 0x7fb7180cdde0 2024-04-15T15:20:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.512+0000 7fb717400640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6f806c980 0x7fb6f806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:20:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.512+0000 7fb717400640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6f806c980 0x7fb6f806ee40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb71815d950 tx=0x7fb704009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:20:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.516+0000 7fb70fe00640 1 -- 172.21.15.49:0/2596388386 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7181f5100 con 0x7fb7180cdde0 2024-04-15T15:20:59.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.720+0000 7fb71d400640 1 -- 172.21.15.49:0/2596388386 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb71806cf30 con 0x7fb7180cdde0 2024-04-15T15:20:59.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.722+0000 7fb70fe00640 1 -- 172.21.15.49:0/2596388386 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+13150 (secure 0 0 0) 0x7fb70805a4c0 con 0x7fb7180cdde0 2024-04-15T15:20:59.724 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:20:59.724 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":39,"fsid":"9ceb114d-b5a6-4a7e-ba1c-c5afe380f71c","created":"2024-04-15T15:18:25.507911+0000","modified":"2024-04-15T15:20:54.120108+0000","last_up_change":"2024-04-15T15:18:37.367735+0000","last_in_change":"2024-04-15T15:18:29.079024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:18:38.806030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13788510-cda8-477d-a73e-0fb27d72cdaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6809","nonce":2011863493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6811","nonce":2011863493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6817","nonce":2011863493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":2011863493},{"type":"v1","addr":"172.21.15.49:6813","nonce":2011863493}]},"public_addr":"172.21.15.49:6809/2011863493","cluster_addr":"172.21.15.49:6811/2011863493","heartbeat_back_addr":"172.21.15.49:6817/2011863493","heartbeat_front_addr":"172.21.15.49:6813/2011863493","state":["exists","up"]},{"osd":1,"uuid":"7c41c3df-3579-4ac0-a310-bd1ec982129a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6801","nonce":4210904815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6803","nonce":4210904815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6807","nonce":4210904815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":4210904815},{"type":"v1","addr":"172.21.15.49:6805","nonce":4210904815}]},"public_addr":"172.21.15.49:6801/4210904815","cluster_addr":"172.21.15.49:6803/4210904815","heartbeat_back_addr":"172.21.15.49:6807/4210904815","heartbeat_front_addr":"172.21.15.49:6805/4210904815","state":["exists","up"]},{"osd":2,"uuid":"e6fd6d93-b856-4958-a445-b88822692f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6816","nonce":1321297224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6819","nonce":1321297224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6825","nonce":1321297224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":1321297224},{"type":"v1","addr":"172.21.15.49:6821","nonce":1321297224}]},"public_addr":"172.21.15.49:6816/1321297224","cluster_addr":"172.21.15.49:6819/1321297224","heartbeat_back_addr":"172.21.15.49:6825/1321297224","heartbeat_front_addr":"172.21.15.49:6821/1321297224","state":["exists","up"]},{"osd":3,"uuid":"eb799a7e-9f21-4a9d-8a9d-f374b58f8074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6824","nonce":1234122534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6827","nonce":1234122534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6831","nonce":1234122534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1234122534},{"type":"v1","addr":"172.21.15.49:6829","nonce":1234122534}]},"public_addr":"172.21.15.49:6824/1234122534","cluster_addr":"172.21.15.49:6827/1234122534","heartbeat_back_addr":"172.21.15.49:6831/1234122534","heartbeat_front_addr":"172.21.15.49:6829/1234122534","state":["exists","up"]},{"osd":4,"uuid":"ae9fb934-8b18-47d9-9ab8-44204f9feb47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6801","nonce":3185658252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6803","nonce":3185658252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6807","nonce":3185658252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3185658252},{"type":"v1","addr":"172.21.15.107:6805","nonce":3185658252}]},"public_addr":"172.21.15.107:6801/3185658252","cluster_addr":"172.21.15.107:6803/3185658252","heartbeat_back_addr":"172.21.15.107:6807/3185658252","heartbeat_front_addr":"172.21.15.107:6805/3185658252","state":["exists","up"]},{"osd":5,"uuid":"f500a3b5-7007-47b2-8d18-3ceecadaa55c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6810","nonce":1708671582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6814","nonce":1708671582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6825","nonce":1708671582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1708671582},{"type":"v1","addr":"172.21.15.107:6819","nonce":1708671582}]},"public_addr":"172.21.15.107:6810/1708671582","cluster_addr":"172.21.15.107:6814/1708671582","heartbeat_back_addr":"172.21.15.107:6825/1708671582","heartbeat_front_addr":"172.21.15.107:6819/1708671582","state":["exists","up"]},{"osd":6,"uuid":"84196c37-bce8-4069-99b8-7368d0798b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6809","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6811","nonce":4081027681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6813","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6815","nonce":4081027681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6821","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6824","nonce":4081027681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6817","nonce":4081027681},{"type":"v1","addr":"172.21.15.107:6818","nonce":4081027681}]},"public_addr":"172.21.15.107:6811/4081027681","cluster_addr":"172.21.15.107:6815/4081027681","heartbeat_back_addr":"172.21.15.107:6824/4081027681","heartbeat_front_addr":"172.21.15.107:6818/4081027681","state":["exists","up"]},{"osd":7,"uuid":"5229bd2f-7a59-43df-b354-92a1648d46a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6823","nonce":876829831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6827","nonce":876829831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6831","nonce":876829831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":876829831},{"type":"v1","addr":"172.21.15.107:6829","nonce":876829831}]},"public_addr":"172.21.15.107:6823/876829831","cluster_addr":"172.21.15.107:6827/876829831","heartbeat_back_addr":"172.21.15.107:6831/876829831","heartbeat_front_addr":"172.21.15.107:6829/876829831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.210454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.426066+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.301391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.529989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.660550+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.477700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.458828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:18:34.563218+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6837/3168788676":"2024-04-16T15:20:51.108477+0000","172.21.15.49:6836/3168788676":"2024-04-16T15:20:51.108477+0000","172.21.15.49:6837/1023024717":"2024-04-16T15:20:28.618766+0000","172.21.15.49:6836/1023024717":"2024-04-16T15:20:28.618766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:20:59.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 -- 172.21.15.49:0/2596388386 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6f806c980 msgr2=0x7fb6f806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6f806c980 0x7fb6f806ee40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb71815d950 tx=0x7fb704009040 comp rx=0 tx=0).stop 2024-04-15T15:20:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 -- 172.21.15.49:0/2596388386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7180cdde0 msgr2=0x7fb7181e6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:20:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7180cdde0 0x7fb7181e6430 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb70800d800 tx=0x7fb70800dcc0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 -- 172.21.15.49:0/2596388386 shutdown_connections 2024-04-15T15:20:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fb6f806c980 0x7fb6f806ee40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb71815b860 0x7fb718146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb71815b480 0x7fb7181f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 --2- 172.21.15.49:0/2596388386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7180cdde0 0x7fb7181e6430 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:20:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 -- 172.21.15.49:0/2596388386 >> 172.21.15.49:0/2596388386 conn(0x7fb71806b930 msgr2=0x7fb71806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:20:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 -- 172.21.15.49:0/2596388386 shutdown_connections 2024-04-15T15:20:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:20:59.725+0000 7fb71d400640 1 -- 172.21.15.49:0/2596388386 wait complete. 2024-04-15T15:21:00.745 INFO:tasks.ceph:Scrubbing osd.0 2024-04-15T15:21:00.745 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:00.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.989+0000 7f5281200640 1 Processor -- start 2024-04-15T15:21:00.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.989+0000 7f5281200640 1 -- start start 2024-04-15T15:21:00.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.991+0000 7f5281200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f527c0cec60 0x7f527c071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:00.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.991+0000 7f5281200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f527c072450 0x7f527c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:00.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.991+0000 7f5281200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f527c15d250 0x7f527c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:00.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.991+0000 7f5281200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f527c13f9e0 con 0x7f527c072450 2024-04-15T15:21:00.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.991+0000 7f5281200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f527c072890 con 0x7f527c15d250 2024-04-15T15:21:00.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.991+0000 7f5281200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f527c13fc70 con 0x7f527c0cec60 2024-04-15T15:21:00.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.991+0000 7f527aa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f527c072450 0x7f527c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:00.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.991+0000 7f527aa00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f527c072450 0x7f527c15cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43640/0 (socket says 172.21.15.49:43640) 2024-04-15T15:21:00.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.991+0000 7f527aa00640 1 -- 172.21.15.49:0/1049102444 learned_addr learned my addr 172.21.15.49:0/1049102444 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:00.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.992+0000 7f527b400640 1 --2- 172.21.15.49:0/1049102444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f527c0cec60 0x7f527c071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:00.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.992+0000 7f527be00640 1 --2- 172.21.15.49:0/1049102444 >> v2:172.21.15.107:3300/0 conn(0x7f527c15d250 0x7f527c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:00.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.992+0000 7f527aa00640 1 -- 172.21.15.49:0/1049102444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f527c0cec60 msgr2=0x7f527c071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:00.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.992+0000 7f527aa00640 1 --2- 172.21.15.49:0/1049102444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f527c0cec60 0x7f527c071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.992+0000 7f527aa00640 1 -- 172.21.15.49:0/1049102444 >> v2:172.21.15.107:3300/0 conn(0x7f527c15d250 msgr2=0x7f527c06f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:00.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.992+0000 7f527aa00640 1 --2- 172.21.15.49:0/1049102444 >> v2:172.21.15.107:3300/0 conn(0x7f527c15d250 0x7f527c06f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.992+0000 7f527aa00640 1 -- 172.21.15.49:0/1049102444 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c15da40 con 0x7f527c072450 2024-04-15T15:21:00.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.992+0000 7f527b400640 1 --2- 172.21.15.49:0/1049102444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f527c0cec60 0x7f527c071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:00.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.993+0000 7f527aa00640 1 --2- 172.21.15.49:0/1049102444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f527c072450 0x7f527c15cd10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f527c072bf0 tx=0x7f526402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74f2ce700e0ff96 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:00.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.994+0000 7f527a000640 1 -- 172.21.15.49:0/1049102444 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264034070 con 0x7f527c072450 2024-04-15T15:21:00.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.994+0000 7f527a000640 1 -- 172.21.15.49:0/1049102444 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5264030850 con 0x7f527c072450 2024-04-15T15:21:00.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.994+0000 7f527a000640 1 -- 172.21.15.49:0/1049102444 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264030c20 con 0x7f527c072450 2024-04-15T15:21:00.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.994+0000 7f5281200640 1 -- 172.21.15.49:0/1049102444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f527c072450 msgr2=0x7f527c15cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:00.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.994+0000 7f5281200640 1 --2- 172.21.15.49:0/1049102444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f527c072450 0x7f527c15cd10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f527c072bf0 tx=0x7f526402b990 comp rx=0 tx=0).stop 2024-04-15T15:21:00.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.994+0000 7f5281200640 1 -- 172.21.15.49:0/1049102444 shutdown_connections 2024-04-15T15:21:00.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.994+0000 7f5281200640 1 --2- 172.21.15.49:0/1049102444 >> v2:172.21.15.107:3300/0 conn(0x7f527c15d250 0x7f527c06f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.995+0000 7f5281200640 1 --2- 172.21.15.49:0/1049102444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f527c072450 0x7f527c15cd10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:00.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.995+0000 7f5281200640 1 --2- 172.21.15.49:0/1049102444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f527c0cec60 0x7f527c071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.995+0000 7f5281200640 1 -- 172.21.15.49:0/1049102444 >> 172.21.15.49:0/1049102444 conn(0x7f527c0b6170 msgr2=0x7f527c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.995+0000 7f5281200640 1 -- 172.21.15.49:0/1049102444 shutdown_connections 2024-04-15T15:21:01.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.995+0000 7f5281200640 1 -- 172.21.15.49:0/1049102444 wait complete. 2024-04-15T15:21:01.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.996+0000 7f5281200640 1 Processor -- start 2024-04-15T15:21:01.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.997+0000 7f5281200640 1 -- start start 2024-04-15T15:21:01.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.997+0000 7f5281200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f527c072450 0x7f527c1527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.997+0000 7f5281200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f527c0cec60 0x7f527c130840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f5281200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f527c15d250 0x7f527c06ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f5281200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f527c1404d0 con 0x7f527c15d250 2024-04-15T15:21:01.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f5281200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f527c1401f0 con 0x7f527c0cec60 2024-04-15T15:21:01.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f5281200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f527c06de40 con 0x7f527c072450 2024-04-15T15:21:01.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f527be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f527c15d250 0x7f527c06ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f527be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f527c15d250 0x7f527c06ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43646/0 (socket says 172.21.15.49:43646) 2024-04-15T15:21:01.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f527be00640 1 -- 172.21.15.49:0/3036045605 learned_addr learned my addr 172.21.15.49:0/3036045605 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:01.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f527aa00640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f527c0cec60 0x7f527c130840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f527b400640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f527c072450 0x7f527c1527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f527be00640 1 -- 172.21.15.49:0/3036045605 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f527c072450 msgr2=0x7f527c1527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f527be00640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f527c072450 0x7f527c1527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f527be00640 1 -- 172.21.15.49:0/3036045605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f527c0cec60 msgr2=0x7f527c130840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f527be00640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f527c0cec60 0x7f527c130840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.998+0000 7f527be00640 1 -- 172.21.15.49:0/3036045605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c1317c0 con 0x7f527c15d250 2024-04-15T15:21:01.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.999+0000 7f527be00640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f527c15d250 0x7f527c06ef70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5274007e60 tx=0x7f527400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.999+0000 7f5273400640 1 -- 172.21.15.49:0/3036045605 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274014420 con 0x7f527c15d250 2024-04-15T15:21:01.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.999+0000 7f5273400640 1 -- 172.21.15.49:0/3036045605 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5274019070 con 0x7f527c15d250 2024-04-15T15:21:01.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.999+0000 7f5273400640 1 -- 172.21.15.49:0/3036045605 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274014820 con 0x7f527c15d250 2024-04-15T15:21:01.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.999+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527c131200 con 0x7f527c15d250 2024-04-15T15:21:01.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:00.999+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f527c12e8c0 con 0x7f527c15d250 2024-04-15T15:21:01.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.000+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f527c068b40 con 0x7f527c15d250 2024-04-15T15:21:01.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.001+0000 7f5273400640 1 -- 172.21.15.49:0/3036045605 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f52740149c0 con 0x7f527c15d250 2024-04-15T15:21:01.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.002+0000 7f5273400640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f525806c9b0 0x7f525806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.002+0000 7f5273400640 1 -- 172.21.15.49:0/3036045605 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f527408c140 con 0x7f527c15d250 2024-04-15T15:21:01.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.003+0000 7f5273400640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] conn(0x7f5258072910 0x7f5258074d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.003+0000 7f5273400640 1 -- 172.21.15.49:0/3036045605 --> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5274014e50 con 0x7f5258072910 2024-04-15T15:21:01.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.003+0000 7f527b400640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f525806c9b0 0x7f525806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.003+0000 7f5273400640 1 -- 172.21.15.49:0/3036045605 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5274093050 con 0x7f527c15d250 2024-04-15T15:21:01.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.003+0000 7f527aa00640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] conn(0x7f5258072910 0x7f5258074d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.003+0000 7f527b400640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f525806c9b0 0x7f525806ee70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f526c006fd0 tx=0x7f526c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.004+0000 7f527aa00640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] conn(0x7f5258072910 0x7f5258074d30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.004+0000 7f5273400640 1 -- 172.21.15.49:0/3036045605 <== osd.0 v2:172.21.15.49:6808/2011863493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5274014e50 con 0x7f5258072910 2024-04-15T15:21:01.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.019+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 --> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f527c0730e0 con 0x7f5258072910 2024-04-15T15:21:01.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.019+0000 7f5273400640 1 -- 172.21.15.49:0/3036045605 <== osd.0 v2:172.21.15.49:6808/2011863493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f527c0730e0 con 0x7f5258072910 2024-04-15T15:21:01.023 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:21:01.023 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:01.023 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:21:01.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.020+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] conn(0x7f5258072910 msgr2=0x7f5258074d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.020+0000 7f5281200640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:6808/2011863493,v1:172.21.15.49:6809/2011863493] conn(0x7f5258072910 0x7f5258074d30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.020+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f525806c9b0 msgr2=0x7f525806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.020+0000 7f5281200640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f525806c9b0 0x7f525806ee70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f526c006fd0 tx=0x7f526c008040 comp rx=0 tx=0).stop 2024-04-15T15:21:01.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.020+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f527c15d250 msgr2=0x7f527c06ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.020+0000 7f5281200640 1 --2- 172.21.15.49:0/3036045605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f527c15d250 0x7f527c06ef70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5274007e60 tx=0x7f527400e4b0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.020+0000 7f527be00640 1 -- 172.21.15.49:0/3036045605 reap_dead start 2024-04-15T15:21:01.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.020+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 shutdown_connections 2024-04-15T15:21:01.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.020+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 >> 172.21.15.49:0/3036045605 conn(0x7f527c0b6170 msgr2=0x7f527c15f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.021+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 shutdown_connections 2024-04-15T15:21:01.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.021+0000 7f5281200640 1 -- 172.21.15.49:0/3036045605 wait complete. 2024-04-15T15:21:01.042 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-15T15:21:01.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.267+0000 7fc0cda00640 1 Processor -- start 2024-04-15T15:21:01.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.267+0000 7fc0cda00640 1 -- start start 2024-04-15T15:21:01.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.269+0000 7fc0cda00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc0c8162e90 0x7fc0c80b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.269+0000 7fc0cda00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0c80afd70 0x7fc0c8072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.269+0000 7fc0cda00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c80b01d0 con 0x7fc0c8072640 2024-04-15T15:21:01.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.269+0000 7fc0cda00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c8072a90 con 0x7fc0c8162e90 2024-04-15T15:21:01.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.269+0000 7fc0cda00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c80b3900 con 0x7fc0c80afd70 2024-04-15T15:21:01.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.269+0000 7fc0c6a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc0c8162e90 0x7fc0c80b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.269+0000 7fc0c7400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc0c8072640 0x7fc0c806e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42500/0 (socket says 172.21.15.49:42500) 2024-04-15T15:21:01.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.269+0000 7fc0c7400640 1 -- 172.21.15.49:0/3634166085 learned_addr learned my addr 172.21.15.49:0/3634166085 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:21:01.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.269+0000 7fc0be000640 1 --2- 172.21.15.49:0/3634166085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0c80afd70 0x7fc0c8072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.270+0000 7fc0c6a00640 1 -- 172.21.15.49:0/3634166085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0c80afd70 msgr2=0x7fc0c8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.270+0000 7fc0c6a00640 1 --2- 172.21.15.49:0/3634166085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0c80afd70 0x7fc0c8072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.270+0000 7fc0c6a00640 1 -- 172.21.15.49:0/3634166085 >> v1:172.21.15.49:6789/0 conn(0x7fc0c8072640 legacy=0x7fc0c806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.270+0000 7fc0c6000640 1 -- 172.21.15.49:0/3634166085 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3965550001 0 0) 0x7fc0c0001490 con 0x7fc0c8072640 2024-04-15T15:21:01.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.270+0000 7fc0c6a00640 1 -- 172.21.15.49:0/3634166085 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c80b3be0 con 0x7fc0c8162e90 2024-04-15T15:21:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.271+0000 7fc0c6a00640 1 --2- 172.21.15.49:0/3634166085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 0x7fc0c80b58e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80097a0 tx=0x7fc0b802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb80fdeb1542509 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.271+0000 7fc0c6000640 1 -- 172.21.15.49:0/3634166085 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80315a0 con 0x7fc0c8162e90 2024-04-15T15:21:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.271+0000 7fc0c6000640 1 -- 172.21.15.49:0/3634166085 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b8037070 con 0x7fc0c8162e90 2024-04-15T15:21:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.271+0000 7fc0c6000640 1 -- 172.21.15.49:0/3634166085 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8031960 con 0x7fc0c8162e90 2024-04-15T15:21:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.272+0000 7fc0cda00640 1 -- 172.21.15.49:0/3634166085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 msgr2=0x7fc0c80b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.272+0000 7fc0cda00640 1 --2- 172.21.15.49:0/3634166085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 0x7fc0c80b58e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80097a0 tx=0x7fc0b802b840 comp rx=0 tx=0).stop 2024-04-15T15:21:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.272+0000 7fc0cda00640 1 -- 172.21.15.49:0/3634166085 shutdown_connections 2024-04-15T15:21:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.272+0000 7fc0cda00640 1 --2- 172.21.15.49:0/3634166085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0c80afd70 0x7fc0c8072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.272+0000 7fc0cda00640 1 --2- 172.21.15.49:0/3634166085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 0x7fc0c80b58e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.272+0000 7fc0cda00640 1 -- 172.21.15.49:0/3634166085 >> 172.21.15.49:0/3634166085 conn(0x7fc0c80b2460 msgr2=0x7fc0c80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.272+0000 7fc0cda00640 1 -- 172.21.15.49:0/3634166085 shutdown_connections 2024-04-15T15:21:01.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.272+0000 7fc0cda00640 1 -- 172.21.15.49:0/3634166085 wait complete. 2024-04-15T15:21:01.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.274+0000 7fc0cda00640 1 Processor -- start 2024-04-15T15:21:01.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.274+0000 7fc0cda00640 1 -- start start 2024-04-15T15:21:01.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.275+0000 7fc0cda00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0c8072640 0x7fc0c81f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.275+0000 7fc0cda00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c80afd70 0x7fc0c81f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.275+0000 7fc0cda00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 0x7fc0c80ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.275+0000 7fc0cda00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c80cf7a0 con 0x7fc0c8162e90 2024-04-15T15:21:01.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.275+0000 7fc0cda00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c80b41a0 con 0x7fc0c80afd70 2024-04-15T15:21:01.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.275+0000 7fc0cda00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c80cf920 con 0x7fc0c8072640 2024-04-15T15:21:01.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.275+0000 7fc0c7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 0x7fc0c80ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0c7400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 0x7fc0c80ab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43658/0 (socket says 172.21.15.49:43658) 2024-04-15T15:21:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0c7400640 1 -- 172.21.15.49:0/2695154190 learned_addr learned my addr 172.21.15.49:0/2695154190 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0c6a00640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0c8072640 0x7fc0c81f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0be000640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c80afd70 0x7fc0c81f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0c7400640 1 -- 172.21.15.49:0/2695154190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0c8072640 msgr2=0x7fc0c81f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0c7400640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0c8072640 0x7fc0c81f91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0c7400640 1 -- 172.21.15.49:0/2695154190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c80afd70 msgr2=0x7fc0c81f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0c7400640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c80afd70 0x7fc0c81f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0c7400640 1 -- 172.21.15.49:0/2695154190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c80a99f0 con 0x7fc0c8162e90 2024-04-15T15:21:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0be000640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c80afd70 0x7fc0c81f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.276+0000 7fc0c7400640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 0x7fc0c80ab990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c700 tx=0x7fc0c000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.277+0000 7fc0bf400640 1 -- 172.21.15.49:0/2695154190 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0018070 con 0x7fc0c8162e90 2024-04-15T15:21:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.277+0000 7fc0bf400640 1 -- 172.21.15.49:0/2695154190 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c000ce10 con 0x7fc0c8162e90 2024-04-15T15:21:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.277+0000 7fc0cda00640 1 -- 172.21.15.49:0/2695154190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c80a9c90 con 0x7fc0c8162e90 2024-04-15T15:21:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.277+0000 7fc0bf400640 1 -- 172.21.15.49:0/2695154190 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000a860 con 0x7fc0c8162e90 2024-04-15T15:21:01.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.277+0000 7fc0cda00640 1 -- 172.21.15.49:0/2695154190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c80aa230 con 0x7fc0c8162e90 2024-04-15T15:21:01.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.278+0000 7fc0cda00640 1 -- 172.21.15.49:0/2695154190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c8068c80 con 0x7fc0c8162e90 2024-04-15T15:21:01.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.279+0000 7fc0bf400640 1 -- 172.21.15.49:0/2695154190 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc0c001b070 con 0x7fc0c8162e90 2024-04-15T15:21:01.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.280+0000 7fc0bf400640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc0a406c980 0x7fc0a406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.280+0000 7fc0bf400640 1 -- 172.21.15.49:0/2695154190 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fc0c008ada0 con 0x7fc0c8162e90 2024-04-15T15:21:01.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.280+0000 7fc0c6a00640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc0a406c980 0x7fc0a406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.281+0000 7fc0c6a00640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc0a406c980 0x7fc0a406ee40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8036000 tx=0x7fc0b803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.285+0000 7fc0bf400640 1 -- 172.21.15.49:0/2695154190 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0c80b01d0 con 0x7fc0c8162e90 2024-04-15T15:21:01.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.487+0000 7fc0cda00640 1 -- 172.21.15.49:0/2695154190 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc0c815d550 con 0x7fc0a406c980 2024-04-15T15:21:01.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.488+0000 7fc0bf400640 1 -- 172.21.15.49:0/2695154190 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc0c815d550 con 0x7fc0a406c980 2024-04-15T15:21:01.491 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 0 to deep-scrub 2024-04-15T15:21:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.491+0000 7fc0cda00640 1 -- 172.21.15.49:0/2695154190 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc0a406c980 msgr2=0x7fc0a406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.491+0000 7fc0cda00640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc0a406c980 0x7fc0a406ee40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8036000 tx=0x7fc0b803d040 comp rx=0 tx=0).stop 2024-04-15T15:21:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.491+0000 7fc0cda00640 1 -- 172.21.15.49:0/2695154190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 msgr2=0x7fc0c80ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.491+0000 7fc0cda00640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 0x7fc0c80ab990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c700 tx=0x7fc0c000cbc0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.492+0000 7fc0cda00640 1 -- 172.21.15.49:0/2695154190 shutdown_connections 2024-04-15T15:21:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.492+0000 7fc0cda00640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fc0a406c980 0x7fc0a406ee40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.492+0000 7fc0cda00640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c8162e90 0x7fc0c80ab990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.492+0000 7fc0cda00640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c80afd70 0x7fc0c81f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.492+0000 7fc0cda00640 1 --2- 172.21.15.49:0/2695154190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0c8072640 0x7fc0c81f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.492+0000 7fc0cda00640 1 -- 172.21.15.49:0/2695154190 >> 172.21.15.49:0/2695154190 conn(0x7fc0c80b2460 msgr2=0x7fc0c8161cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.492+0000 7fc0cda00640 1 -- 172.21.15.49:0/2695154190 shutdown_connections 2024-04-15T15:21:01.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.492+0000 7fc0cda00640 1 -- 172.21.15.49:0/2695154190 wait complete. 2024-04-15T15:21:01.515 INFO:tasks.ceph:Scrubbing osd.1 2024-04-15T15:21:01.516 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:01.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.749+0000 7f0601a00640 1 Processor -- start 2024-04-15T15:21:01.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.750+0000 7f0601a00640 1 -- start start 2024-04-15T15:21:01.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.751+0000 7f0601a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc144e40 0x7f05fc06ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.751+0000 7f0601a00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05fc0b3cf0 con 0x7f05fc0cdb80 2024-04-15T15:21:01.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.751+0000 7f0601a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05fc0b3fd0 con 0x7f05fc06f490 2024-04-15T15:21:01.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.751+0000 7f0601a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f05fc06f8d0 con 0x7f05fc144e40 2024-04-15T15:21:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.752+0000 7f0601000640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f05fc0cdb80 0x7f05fc15bad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42508/0 (socket says 172.21.15.49:42508) 2024-04-15T15:21:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.752+0000 7f0601000640 1 -- 172.21.15.49:0/2297710998 learned_addr learned my addr 172.21.15.49:0/2297710998 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:21:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.752+0000 7f05fbe00640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc144e40 0x7f05fc06ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.752+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1123974731 0 0) 0x7f05fc0b3cf0 con 0x7f05fc0cdb80 2024-04-15T15:21:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.752+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05dc003520 con 0x7f05fc0cdb80 2024-04-15T15:21:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4108969133 0 0) 0x7f05dc003520 con 0x7f05fc0cdb80 2024-04-15T15:21:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc144e40 msgr2=0x7f05fc06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc144e40 0x7f05fc06ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 >> v1:172.21.15.107:6789/0 conn(0x7f05fc06f490 legacy=0x7f05fc0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc0b42b0 con 0x7f05fc0cdb80 2024-04-15T15:21:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f05f40019f0 con 0x7f05fc0cdb80 2024-04-15T15:21:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 >> v1:172.21.15.49:6789/0 conn(0x7f05fc0cdb80 legacy=0x7f05fc15bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05dc0039e0 0x7f05dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc144e40 0x7f05dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 0x7f05dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc06f8d0 con 0x7f05dc004d40 2024-04-15T15:21:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc00dd00 con 0x7f05dc0039e0 2024-04-15T15:21:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f05dc00df00 con 0x7f05fc144e40 2024-04-15T15:21:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.753+0000 7f0601000640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 0x7f05dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.754+0000 7f05fb400640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc144e40 0x7f05dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.754+0000 7f05fbe00640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05dc0039e0 0x7f05dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.754+0000 7f0601000640 1 -- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc144e40 msgr2=0x7f05dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.754+0000 7f0601000640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc144e40 0x7f05dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.754+0000 7f0601000640 1 -- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05dc0039e0 msgr2=0x7f05dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.754+0000 7f0601000640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05dc0039e0 0x7f05dc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.754+0000 7f0601000640 1 -- 172.21.15.49:0/2297710998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc003520 con 0x7f05dc004d40 2024-04-15T15:21:01.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.754+0000 7f05fbe00640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05dc0039e0 0x7f05dc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.755+0000 7f0601000640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 0x7f05dc0051c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f05f400a840 tx=0x7f05dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08e5f88a40ddd5f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.755+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f400a460 con 0x7f05dc004d40 2024-04-15T15:21:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.755+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f4034650 con 0x7f05dc004d40 2024-04-15T15:21:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.755+0000 7f05faa00640 1 -- 172.21.15.49:0/2297710998 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4034a40 con 0x7f05dc004d40 2024-04-15T15:21:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.756+0000 7f0601a00640 1 -- 172.21.15.49:0/2297710998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 msgr2=0x7f05dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.756+0000 7f0601a00640 1 --2- 172.21.15.49:0/2297710998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 0x7f05dc0051c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f05f400a840 tx=0x7f05dc005830 comp rx=0 tx=0).stop 2024-04-15T15:21:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.756+0000 7f0601000640 1 -- 172.21.15.49:0/2297710998 reap_dead start 2024-04-15T15:21:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.756+0000 7f0601a00640 1 -- 172.21.15.49:0/2297710998 shutdown_connections 2024-04-15T15:21:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.756+0000 7f0601a00640 1 -- 172.21.15.49:0/2297710998 >> 172.21.15.49:0/2297710998 conn(0x7f05fc0b6170 msgr2=0x7f05fc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.756+0000 7f0601a00640 1 -- 172.21.15.49:0/2297710998 shutdown_connections 2024-04-15T15:21:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.756+0000 7f0601a00640 1 -- 172.21.15.49:0/2297710998 wait complete. 2024-04-15T15:21:01.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.758+0000 7f0601a00640 1 Processor -- start 2024-04-15T15:21:01.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.758+0000 7f0601a00640 1 -- start start 2024-04-15T15:21:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.759+0000 7f0601a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 0x7f05fc0c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.759+0000 7f0601a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0cc2c0 0x7f05fc1321c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.759+0000 7f0601a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc0cc720 0x7f05fc132700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.759+0000 7f0601a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc145220 con 0x7f05dc004d40 2024-04-15T15:21:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.759+0000 7f0601a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc0b4870 con 0x7f05fc0cc720 2024-04-15T15:21:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.759+0000 7f0601a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f05fc0b4d90 con 0x7f05fc0cc2c0 2024-04-15T15:21:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.759+0000 7f05fbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 0x7f05fc0c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.759+0000 7f05fbe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 0x7f05fc0c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43686/0 (socket says 172.21.15.49:43686) 2024-04-15T15:21:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.759+0000 7f05fbe00640 1 -- 172.21.15.49:0/3100216287 learned_addr learned my addr 172.21.15.49:0/3100216287 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f05fb400640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0cc2c0 0x7f05fc1321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f05fbe00640 1 -- 172.21.15.49:0/3100216287 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0cc2c0 msgr2=0x7f05fc1321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f05fbe00640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0cc2c0 0x7f05fc1321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f05fbe00640 1 -- 172.21.15.49:0/3100216287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc0cc720 msgr2=0x7f05fc132700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f05fbe00640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc0cc720 0x7f05fc132700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f05fbe00640 1 -- 172.21.15.49:0/3100216287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc131120 con 0x7f05dc004d40 2024-04-15T15:21:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f05fb400640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0cc2c0 0x7f05fc1321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f05fbe00640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 0x7f05fc0c99d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0108c0 tx=0x7f05ec010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f05f3e00640 1 -- 172.21.15.49:0/3100216287 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec00f040 con 0x7f05dc004d40 2024-04-15T15:21:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f05f3e00640 1 -- 172.21.15.49:0/3100216287 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05ec00ea80 con 0x7f05dc004d40 2024-04-15T15:21:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.760+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc131400 con 0x7f05dc004d40 2024-04-15T15:21:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.761+0000 7f05f3e00640 1 -- 172.21.15.49:0/3100216287 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec00ee50 con 0x7f05dc004d40 2024-04-15T15:21:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.761+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1319c0 con 0x7f05dc004d40 2024-04-15T15:21:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.761+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f05fc130ed0 con 0x7f05dc004d40 2024-04-15T15:21:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.763+0000 7f05f3e00640 1 -- 172.21.15.49:0/3100216287 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f05ec01c400 con 0x7f05dc004d40 2024-04-15T15:21:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.764+0000 7f05f3e00640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f05d806c9b0 0x7f05d806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.764+0000 7f05f3e00640 1 -- 172.21.15.49:0/3100216287 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f05ec08c820 con 0x7f05dc004d40 2024-04-15T15:21:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.764+0000 7f05f3e00640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] conn(0x7f05d8072910 0x7f05d8074d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.764+0000 7f05fb400640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f05d806c9b0 0x7f05d806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.764+0000 7f05f3e00640 1 -- 172.21.15.49:0/3100216287 --> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05ec004470 con 0x7f05d8072910 2024-04-15T15:21:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.764+0000 7f05f3e00640 1 -- 172.21.15.49:0/3100216287 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f05ec08cbd0 con 0x7f05dc004d40 2024-04-15T15:21:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.764+0000 7f0601000640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] conn(0x7f05d8072910 0x7f05d8074d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.765+0000 7f05fb400640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f05d806c9b0 0x7f05d806ee70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0b3310 tx=0x7f05e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.765+0000 7f0601000640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] conn(0x7f05d8072910 0x7f05d8074d30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.766+0000 7f05f3e00640 1 -- 172.21.15.49:0/3100216287 <== osd.1 v2:172.21.15.49:6800/4210904815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05ec004470 con 0x7f05d8072910 2024-04-15T15:21:01.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.781+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 --> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f05fc068b40 con 0x7f05d8072910 2024-04-15T15:21:01.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.781+0000 7f05f3e00640 1 -- 172.21.15.49:0/3100216287 <== osd.1 v2:172.21.15.49:6800/4210904815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f05fc068b40 con 0x7f05d8072910 2024-04-15T15:21:01.785 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:21:01.785 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:01.785 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:21:01.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.782+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] conn(0x7f05d8072910 msgr2=0x7f05d8074d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.782+0000 7f0601a00640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:6800/4210904815,v1:172.21.15.49:6801/4210904815] conn(0x7f05d8072910 0x7f05d8074d30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:01.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.782+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f05d806c9b0 msgr2=0x7f05d806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.782+0000 7f0601a00640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f05d806c9b0 0x7f05d806ee70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0b3310 tx=0x7f05e4009040 comp rx=0 tx=0).stop 2024-04-15T15:21:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.782+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 msgr2=0x7f05fc0c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.782+0000 7f0601a00640 1 --2- 172.21.15.49:0/3100216287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05dc004d40 0x7f05fc0c99d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0108c0 tx=0x7f05ec010d80 comp rx=0 tx=0).stop 2024-04-15T15:21:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.782+0000 7f0601000640 1 -- 172.21.15.49:0/3100216287 reap_dead start 2024-04-15T15:21:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.782+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 shutdown_connections 2024-04-15T15:21:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.782+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 >> 172.21.15.49:0/3100216287 conn(0x7f05fc0b6170 msgr2=0x7f05fc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.783+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 shutdown_connections 2024-04-15T15:21:01.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:01.783+0000 7f0601a00640 1 -- 172.21.15.49:0/3100216287 wait complete. 2024-04-15T15:21:01.804 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-15T15:21:02.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.029+0000 7fe52a400640 1 Processor -- start 2024-04-15T15:21:02.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.030+0000 7fe52a400640 1 -- start start 2024-04-15T15:21:02.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.031+0000 7fe52a400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe5240b0cb0 0x7fe5240ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.031+0000 7fe52a400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe52415d770 0x7fe5240cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.032+0000 7fe52a400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5240cd940 con 0x7fe5240b59b0 2024-04-15T15:21:02.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.032+0000 7fe52a400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fe5240ce550 con 0x7fe5240b0cb0 2024-04-15T15:21:02.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.032+0000 7fe52a400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe5240cdc20 con 0x7fe52415d770 2024-04-15T15:21:02.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.032+0000 7fe529a00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe5240b59b0 0x7fe52415dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42512/0 (socket says 172.21.15.49:42512) 2024-04-15T15:21:02.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.032+0000 7fe529a00640 1 -- 172.21.15.49:0/3449170516 learned_addr learned my addr 172.21.15.49:0/3449170516 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:21:02.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.032+0000 7fe523e00640 1 --2- 172.21.15.49:0/3449170516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe52415d770 0x7fe5240cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.032+0000 7fe529000640 1 --2- 172.21.15.49:0/3449170516 >> v2:172.21.15.107:3300/0 conn(0x7fe5240b0cb0 0x7fe5240ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.033+0000 7fe523400640 1 -- 172.21.15.49:0/3449170516 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494356877 0 0) 0x7fe5240cd940 con 0x7fe5240b59b0 2024-04-15T15:21:02.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.033+0000 7fe523400640 1 -- 172.21.15.49:0/3449170516 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe508003520 con 0x7fe5240b59b0 2024-04-15T15:21:02.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.033+0000 7fe529000640 1 -- 172.21.15.49:0/3449170516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe52415d770 msgr2=0x7fe5240cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.033+0000 7fe529000640 1 --2- 172.21.15.49:0/3449170516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe52415d770 0x7fe5240cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.033+0000 7fe529000640 1 -- 172.21.15.49:0/3449170516 >> v1:172.21.15.49:6789/0 conn(0x7fe5240b59b0 legacy=0x7fe52415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.033+0000 7fe529000640 1 -- 172.21.15.49:0/3449170516 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5240cdf00 con 0x7fe5240b0cb0 2024-04-15T15:21:02.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.034+0000 7fe529000640 1 --2- 172.21.15.49:0/3449170516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5240b0cb0 0x7fe5240ce9e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe5140097a0 tx=0x7fe51402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b12f75e0810cb755 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.035+0000 7fe523400640 1 -- 172.21.15.49:0/3449170516 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514035020 con 0x7fe5240b0cb0 2024-04-15T15:21:02.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.035+0000 7fe523400640 1 -- 172.21.15.49:0/3449170516 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe514031810 con 0x7fe5240b0cb0 2024-04-15T15:21:02.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.035+0000 7fe523400640 1 -- 172.21.15.49:0/3449170516 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514031be0 con 0x7fe5240b0cb0 2024-04-15T15:21:02.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.035+0000 7fe52a400640 1 -- 172.21.15.49:0/3449170516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5240b0cb0 msgr2=0x7fe5240ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.035+0000 7fe52a400640 1 --2- 172.21.15.49:0/3449170516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5240b0cb0 0x7fe5240ce9e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe5140097a0 tx=0x7fe51402b8b0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.036+0000 7fe52a400640 1 -- 172.21.15.49:0/3449170516 shutdown_connections 2024-04-15T15:21:02.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.036+0000 7fe52a400640 1 --2- 172.21.15.49:0/3449170516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe52415d770 0x7fe5240cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.036+0000 7fe52a400640 1 --2- 172.21.15.49:0/3449170516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5240b0cb0 0x7fe5240ce9e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.036+0000 7fe52a400640 1 -- 172.21.15.49:0/3449170516 >> 172.21.15.49:0/3449170516 conn(0x7fe5240b23a0 msgr2=0x7fe5240b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.036+0000 7fe52a400640 1 -- 172.21.15.49:0/3449170516 shutdown_connections 2024-04-15T15:21:02.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.036+0000 7fe52a400640 1 -- 172.21.15.49:0/3449170516 wait complete. 2024-04-15T15:21:02.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.037+0000 7fe52a400640 1 Processor -- start 2024-04-15T15:21:02.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.038+0000 7fe52a400640 1 -- start start 2024-04-15T15:21:02.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe52a400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5240b0cb0 0x7fe524156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe52a400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5240b59b0 0x7fe52414bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe52a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52415d770 0x7fe52415e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe52a400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5240cef20 con 0x7fe5240b59b0 2024-04-15T15:21:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe52a400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5240b1090 con 0x7fe52415d770 2024-04-15T15:21:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe52a400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe52415ed00 con 0x7fe5240b0cb0 2024-04-15T15:21:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe523e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5240b59b0 0x7fe52414bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe523e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5240b59b0 0x7fe52414bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43692/0 (socket says 172.21.15.49:43692) 2024-04-15T15:21:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe523e00640 1 -- 172.21.15.49:0/1152293265 learned_addr learned my addr 172.21.15.49:0/1152293265 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:02.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe529000640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5240b0cb0 0x7fe524156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe529a00640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52415d770 0x7fe52415e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe523e00640 1 -- 172.21.15.49:0/1152293265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5240b0cb0 msgr2=0x7fe524156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe523e00640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5240b0cb0 0x7fe524156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe523e00640 1 -- 172.21.15.49:0/1152293265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52415d770 msgr2=0x7fe52415e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe523e00640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52415d770 0x7fe52415e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.039+0000 7fe523e00640 1 -- 172.21.15.49:0/1152293265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52415ea60 con 0x7fe5240b59b0 2024-04-15T15:21:02.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.040+0000 7fe523e00640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5240b59b0 0x7fe52414bdf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00e8a0 tx=0x7fe50c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.040+0000 7fe520c00640 1 -- 172.21.15.49:0/1152293265 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c017070 con 0x7fe5240b59b0 2024-04-15T15:21:02.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.040+0000 7fe520c00640 1 -- 172.21.15.49:0/1152293265 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe50c00c940 con 0x7fe5240b59b0 2024-04-15T15:21:02.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.040+0000 7fe520c00640 1 -- 172.21.15.49:0/1152293265 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c00cd30 con 0x7fe5240b59b0 2024-04-15T15:21:02.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.040+0000 7fe52a400640 1 -- 172.21.15.49:0/1152293265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe524152be0 con 0x7fe5240b59b0 2024-04-15T15:21:02.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.040+0000 7fe52a400640 1 -- 172.21.15.49:0/1152293265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe524158d70 con 0x7fe5240b59b0 2024-04-15T15:21:02.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.042+0000 7fe52a400640 1 -- 172.21.15.49:0/1152293265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5240755c0 con 0x7fe5240b59b0 2024-04-15T15:21:02.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.042+0000 7fe520c00640 1 -- 172.21.15.49:0/1152293265 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe50c01a070 con 0x7fe5240b59b0 2024-04-15T15:21:02.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.043+0000 7fe520c00640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe50006c9b0 0x7fe50006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.044+0000 7fe520c00640 1 -- 172.21.15.49:0/1152293265 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fe50c08b730 con 0x7fe5240b59b0 2024-04-15T15:21:02.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.044+0000 7fe529000640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe50006c9b0 0x7fe50006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.045+0000 7fe529000640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe50006c9b0 0x7fe50006ee70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe514036000 tx=0x7fe51403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.048+0000 7fe520c00640 1 -- 172.21.15.49:0/1152293265 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe50c059e10 con 0x7fe5240b59b0 2024-04-15T15:21:02.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.256+0000 7fe52a400640 1 -- 172.21.15.49:0/1152293265 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fe524152990 con 0x7fe50006c9b0 2024-04-15T15:21:02.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.257+0000 7fe520c00640 1 -- 172.21.15.49:0/1152293265 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe524152990 con 0x7fe50006c9b0 2024-04-15T15:21:02.260 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 1 to deep-scrub 2024-04-15T15:21:02.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.259+0000 7fe52a400640 1 -- 172.21.15.49:0/1152293265 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe50006c9b0 msgr2=0x7fe50006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.259+0000 7fe52a400640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe50006c9b0 0x7fe50006ee70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe514036000 tx=0x7fe51403c040 comp rx=0 tx=0).stop 2024-04-15T15:21:02.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.259+0000 7fe52a400640 1 -- 172.21.15.49:0/1152293265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5240b59b0 msgr2=0x7fe52414bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.259+0000 7fe52a400640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5240b59b0 0x7fe52414bdf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00e8a0 tx=0x7fe50c00ed60 comp rx=0 tx=0).stop 2024-04-15T15:21:02.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.260+0000 7fe52a400640 1 -- 172.21.15.49:0/1152293265 shutdown_connections 2024-04-15T15:21:02.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.260+0000 7fe52a400640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe50006c9b0 0x7fe50006ee70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.260+0000 7fe52a400640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52415d770 0x7fe52415e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.260+0000 7fe52a400640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5240b59b0 0x7fe52414bdf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.260+0000 7fe52a400640 1 --2- 172.21.15.49:0/1152293265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5240b0cb0 0x7fe524156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.260+0000 7fe52a400640 1 -- 172.21.15.49:0/1152293265 >> 172.21.15.49:0/1152293265 conn(0x7fe5240b23a0 msgr2=0x7fe5240af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.260+0000 7fe52a400640 1 -- 172.21.15.49:0/1152293265 shutdown_connections 2024-04-15T15:21:02.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.260+0000 7fe52a400640 1 -- 172.21.15.49:0/1152293265 wait complete. 2024-04-15T15:21:02.282 INFO:tasks.ceph:Scrubbing osd.2 2024-04-15T15:21:02.282 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:02.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.516+0000 7f8987200640 1 Processor -- start 2024-04-15T15:21:02.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.517+0000 7f8987200640 1 -- start start 2024-04-15T15:21:02.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.518+0000 7f8987200640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8988072620 0x7f898815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.518+0000 7f8987200640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f898815d250 0x7f898806f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.518+0000 7f8987200640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f898813fc70 con 0x7f89880cec60 2024-04-15T15:21:02.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.518+0000 7f8987200640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f898813f9e0 con 0x7f898815d250 2024-04-15T15:21:02.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.518+0000 7f8987200640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f8988141a00 con 0x7f8988072620 2024-04-15T15:21:02.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.518+0000 7f8985e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f89880cec60 0x7f8988071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42522/0 (socket says 172.21.15.49:42522) 2024-04-15T15:21:02.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.518+0000 7f8985e00640 1 -- 172.21.15.49:0/573233592 learned_addr learned my addr 172.21.15.49:0/573233592 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:21:02.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.518+0000 7f8986800640 1 --2- 172.21.15.49:0/573233592 >> v2:172.21.15.49:3300/0 conn(0x7f898815d250 0x7f898806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.519+0000 7f897fe00640 1 --2- 172.21.15.49:0/573233592 >> v2:172.21.15.107:3300/0 conn(0x7f8988072620 0x7f898815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.519+0000 7f8986800640 1 -- 172.21.15.49:0/573233592 >> v2:172.21.15.107:3300/0 conn(0x7f8988072620 msgr2=0x7f898815cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.519+0000 7f8986800640 1 --2- 172.21.15.49:0/573233592 >> v2:172.21.15.107:3300/0 conn(0x7f8988072620 0x7f898815cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.519+0000 7f8986800640 1 -- 172.21.15.49:0/573233592 >> v1:172.21.15.49:6789/0 conn(0x7f89880cec60 legacy=0x7f8988071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.519+0000 7f8986800640 1 -- 172.21.15.49:0/573233592 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898815da40 con 0x7f898815d250 2024-04-15T15:21:02.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.519+0000 7f8985400640 1 -- 172.21.15.49:0/573233592 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441225843 0 0) 0x7f898813fc70 con 0x7f89880cec60 2024-04-15T15:21:02.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.520+0000 7f8986800640 1 --2- 172.21.15.49:0/573233592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f898815d250 0x7f898806f110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8980009d90 tx=0x7f898002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4244efaa71dcb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.521+0000 7f8985400640 1 -- 172.21.15.49:0/573233592 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980039070 con 0x7f898815d250 2024-04-15T15:21:02.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.521+0000 7f8985400640 1 -- 172.21.15.49:0/573233592 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f898002cd00 con 0x7f898815d250 2024-04-15T15:21:02.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.521+0000 7f8985400640 1 -- 172.21.15.49:0/573233592 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980034660 con 0x7f898815d250 2024-04-15T15:21:02.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.521+0000 7f8987200640 1 -- 172.21.15.49:0/573233592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f898815d250 msgr2=0x7f898806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.521+0000 7f8987200640 1 --2- 172.21.15.49:0/573233592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f898815d250 0x7f898806f110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8980009d90 tx=0x7f898002bf50 comp rx=0 tx=0).stop 2024-04-15T15:21:02.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.521+0000 7f8987200640 1 -- 172.21.15.49:0/573233592 shutdown_connections 2024-04-15T15:21:02.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.521+0000 7f8987200640 1 --2- 172.21.15.49:0/573233592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f898815d250 0x7f898806f110 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.521+0000 7f8987200640 1 --2- 172.21.15.49:0/573233592 >> v2:172.21.15.107:3300/0 conn(0x7f8988072620 0x7f898815cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.521+0000 7f8987200640 1 -- 172.21.15.49:0/573233592 >> 172.21.15.49:0/573233592 conn(0x7f89880b6170 msgr2=0x7f898806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.522+0000 7f8987200640 1 -- 172.21.15.49:0/573233592 shutdown_connections 2024-04-15T15:21:02.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.522+0000 7f8987200640 1 -- 172.21.15.49:0/573233592 wait complete. 2024-04-15T15:21:02.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.523+0000 7f8987200640 1 Processor -- start 2024-04-15T15:21:02.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.524+0000 7f8987200640 1 -- start start 2024-04-15T15:21:02.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.524+0000 7f8987200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988072620 0x7f8988152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.524+0000 7f8987200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89880cec60 0x7f8988153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.524+0000 7f8987200640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f898815d250 0x7f8988154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.524+0000 7f8987200640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f89881404d0 con 0x7f8988072620 2024-04-15T15:21:02.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.524+0000 7f8987200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89881401f0 con 0x7f89880cec60 2024-04-15T15:21:02.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.524+0000 7f8987200640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f898806de40 con 0x7f898815d250 2024-04-15T15:21:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.525+0000 7f8985e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988072620 0x7f8988152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.525+0000 7f8985e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988072620 0x7f8988152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43726/0 (socket says 172.21.15.49:43726) 2024-04-15T15:21:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.525+0000 7f8985e00640 1 -- 172.21.15.49:0/902207281 learned_addr learned my addr 172.21.15.49:0/902207281 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.525+0000 7f8986800640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f898815d250 0x7f8988154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.525+0000 7f897fe00640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89880cec60 0x7f8988153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.525+0000 7f8985e00640 1 -- 172.21.15.49:0/902207281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f898815d250 msgr2=0x7f8988154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.525+0000 7f8985e00640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f898815d250 0x7f8988154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.525+0000 7f8985e00640 1 -- 172.21.15.49:0/902207281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89880cec60 msgr2=0x7f8988153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.525+0000 7f8985e00640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89880cec60 0x7f8988153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.525+0000 7f8985e00640 1 -- 172.21.15.49:0/902207281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89880b3020 con 0x7f8988072620 2024-04-15T15:21:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.526+0000 7f8985e00640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988072620 0x7f8988152840 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f897800c4d0 tx=0x7f897800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.526+0000 7f897e000640 1 -- 172.21.15.49:0/902207281 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978018070 con 0x7f8988072620 2024-04-15T15:21:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.526+0000 7f897e000640 1 -- 172.21.15.49:0/902207281 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8978007500 con 0x7f8988072620 2024-04-15T15:21:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.526+0000 7f897e000640 1 -- 172.21.15.49:0/902207281 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978004a20 con 0x7f8988072620 2024-04-15T15:21:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.526+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89880ad8b0 con 0x7f8988072620 2024-04-15T15:21:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.526+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89880ade70 con 0x7f8988072620 2024-04-15T15:21:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.527+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8988152d80 con 0x7f8988072620 2024-04-15T15:21:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.529+0000 7f897e000640 1 -- 172.21.15.49:0/902207281 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8978004bc0 con 0x7f8988072620 2024-04-15T15:21:02.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.530+0000 7f897e000640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f896406c980 0x7f896406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.530+0000 7f897e000640 1 -- 172.21.15.49:0/902207281 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f897808ab10 con 0x7f8988072620 2024-04-15T15:21:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.530+0000 7f897fe00640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f896406c980 0x7f896406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.530+0000 7f897e000640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] conn(0x7f89640728e0 0x7f8964074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.530+0000 7f897e000640 1 -- 172.21.15.49:0/902207281 --> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89640753b0 con 0x7f89640728e0 2024-04-15T15:21:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.530+0000 7f897e000640 1 -- 172.21.15.49:0/902207281 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f897808f2c0 con 0x7f8988072620 2024-04-15T15:21:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.530+0000 7f8986800640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] conn(0x7f89640728e0 0x7f8964074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.531+0000 7f897fe00640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f896406c980 0x7f896406ee40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89881421c0 tx=0x7f8970008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.531+0000 7f8986800640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] conn(0x7f89640728e0 0x7f8964074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.532+0000 7f897e000640 1 -- 172.21.15.49:0/902207281 <== osd.2 v2:172.21.15.49:6814/1321297224 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89640753b0 con 0x7f89640728e0 2024-04-15T15:21:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.547+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 --> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f89880ae590 con 0x7f89640728e0 2024-04-15T15:21:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.547+0000 7f897e000640 1 -- 172.21.15.49:0/902207281 <== osd.2 v2:172.21.15.49:6814/1321297224 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f89880ae590 con 0x7f89640728e0 2024-04-15T15:21:02.551 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:21:02.551 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:02.551 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:21:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] conn(0x7f89640728e0 msgr2=0x7f8964074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8987200640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:6814/1321297224,v1:172.21.15.49:6816/1321297224] conn(0x7f89640728e0 0x7f8964074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f896406c980 msgr2=0x7f896406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8987200640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f896406c980 0x7f896406ee40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89881421c0 tx=0x7f8970008040 comp rx=0 tx=0).stop 2024-04-15T15:21:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988072620 msgr2=0x7f8988152840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8987200640 1 --2- 172.21.15.49:0/902207281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988072620 0x7f8988152840 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f897800c4d0 tx=0x7f897800c990 comp rx=0 tx=0).stop 2024-04-15T15:21:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8986800640 1 -- 172.21.15.49:0/902207281 reap_dead start 2024-04-15T15:21:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 shutdown_connections 2024-04-15T15:21:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 >> 172.21.15.49:0/902207281 conn(0x7f89880b6170 msgr2=0x7f898806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 shutdown_connections 2024-04-15T15:21:02.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.548+0000 7f8987200640 1 -- 172.21.15.49:0/902207281 wait complete. 2024-04-15T15:21:02.569 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-15T15:21:02.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.794+0000 7f12a9e00640 1 Processor -- start 2024-04-15T15:21:02.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.795+0000 7f12a9e00640 1 -- start start 2024-04-15T15:21:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.796+0000 7f12a9e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f12a406d590 0x7f12a40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.796+0000 7f12a9e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12a40d4650 con 0x7f12a41620b0 2024-04-15T15:21:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.796+0000 7f12a9e00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12a40d2ea0 con 0x7f12a40cf7f0 2024-04-15T15:21:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.796+0000 7f12a9e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f12a40d3180 con 0x7f12a406d590 2024-04-15T15:21:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.796+0000 7f12a8a00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f12a41620b0 0x7f12a40d3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42528/0 (socket says 172.21.15.49:42528) 2024-04-15T15:21:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.796+0000 7f12a8a00640 1 -- 172.21.15.49:0/3132348998 learned_addr learned my addr 172.21.15.49:0/3132348998 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:21:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.797+0000 7f12a9400640 1 --2- 172.21.15.49:0/3132348998 >> v2:172.21.15.107:3300/0 conn(0x7f12a406d590 0x7f12a40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.797+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165256977 0 0) 0x7f12a40d4650 con 0x7f12a41620b0 2024-04-15T15:21:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.797+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f128c003520 con 0x7f12a41620b0 2024-04-15T15:21:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.797+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3905855431 0 0) 0x7f128c003520 con 0x7f12a41620b0 2024-04-15T15:21:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 >> v2:172.21.15.107:3300/0 conn(0x7f12a406d590 msgr2=0x7f12a40b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 --2- 172.21.15.49:0/3132348998 >> v2:172.21.15.107:3300/0 conn(0x7f12a406d590 0x7f12a40b79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 >> v1:172.21.15.107:6789/0 conn(0x7f12a40cf7f0 legacy=0x7f12a406ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a40d3460 con 0x7f12a41620b0 2024-04-15T15:21:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f1298001670 con 0x7f12a41620b0 2024-04-15T15:21:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 >> v1:172.21.15.49:6789/0 conn(0x7f12a41620b0 legacy=0x7f12a40d3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f128c0039b0 0x7f128c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f128c004800 0x7f128c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f128c0051c0 0x7f128c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12a40d3180 con 0x7f128c0051c0 2024-04-15T15:21:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f128c00c070 con 0x7f128c004800 2024-04-15T15:21:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f128c00c270 con 0x7f128c0039b0 2024-04-15T15:21:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a9400640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f128c0051c0 0x7f128c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a8a00640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f128c0039b0 0x7f128c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.798+0000 7f12a3e00640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f128c004800 0x7f128c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.799+0000 7f12a9400640 1 -- 172.21.15.49:0/3132348998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f128c0039b0 msgr2=0x7f128c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.799+0000 7f12a9400640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f128c0039b0 0x7f128c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.799+0000 7f12a9400640 1 -- 172.21.15.49:0/3132348998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f128c004800 msgr2=0x7f128c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.799+0000 7f12a9400640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f128c004800 0x7f128c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.799+0000 7f12a9400640 1 -- 172.21.15.49:0/3132348998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c003520 con 0x7f128c0051c0 2024-04-15T15:21:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.800+0000 7f12a9400640 1 -- 172.21.15.49:0/3132348998 reap_dead start 2024-04-15T15:21:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.800+0000 7f12a9400640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f128c0051c0 0x7f128c00b9c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f12a40d2380 tx=0x7f128c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768bbd39463ae0f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.800+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c039070 con 0x7f128c0051c0 2024-04-15T15:21:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.800+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129c02cbd0 con 0x7f128c0051c0 2024-04-15T15:21:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.800+0000 7f12a3400640 1 -- 172.21.15.49:0/3132348998 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c009c10 con 0x7f128c0051c0 2024-04-15T15:21:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.801+0000 7f12a9e00640 1 -- 172.21.15.49:0/3132348998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f128c0051c0 msgr2=0x7f128c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.801+0000 7f12a9e00640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f128c0051c0 0x7f128c00b9c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f12a40d2380 tx=0x7f128c005830 comp rx=0 tx=0).stop 2024-04-15T15:21:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.801+0000 7f12a9e00640 1 -- 172.21.15.49:0/3132348998 shutdown_connections 2024-04-15T15:21:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.801+0000 7f12a9e00640 1 --2- 172.21.15.49:0/3132348998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f128c0051c0 0x7f128c00b9c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.801+0000 7f12a9e00640 1 -- 172.21.15.49:0/3132348998 >> 172.21.15.49:0/3132348998 conn(0x7f12a40d18d0 msgr2=0x7f12a40d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.801+0000 7f12a9e00640 1 -- 172.21.15.49:0/3132348998 shutdown_connections 2024-04-15T15:21:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.801+0000 7f12a9e00640 1 -- 172.21.15.49:0/3132348998 wait complete. 2024-04-15T15:21:02.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.803+0000 7f12a9e00640 1 Processor -- start 2024-04-15T15:21:02.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.803+0000 7f12a9e00640 1 -- start start 2024-04-15T15:21:02.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.804+0000 7f12a9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f128c0051c0 0x7f12a4160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.804+0000 7f12a9e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12a414aed0 0x7f12a40b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.804+0000 7f12a9e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12a40b9390 0x7f12a41f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.804+0000 7f12a9e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4162490 con 0x7f12a414aed0 2024-04-15T15:21:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.804+0000 7f12a9e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12a40d3a20 con 0x7f128c0051c0 2024-04-15T15:21:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.804+0000 7f12a9e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f12a40d5550 con 0x7f12a40b9390 2024-04-15T15:21:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.804+0000 7f12a3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12a414aed0 0x7f12a40b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.804+0000 7f12a3e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12a414aed0 0x7f12a40b8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43744/0 (socket says 172.21.15.49:43744) 2024-04-15T15:21:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.804+0000 7f12a3e00640 1 -- 172.21.15.49:0/2850671935 learned_addr learned my addr 172.21.15.49:0/2850671935 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.805+0000 7f12a9400640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12a40b9390 0x7f12a41f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.805+0000 7f12a8a00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f128c0051c0 0x7f12a4160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.805+0000 7f12a3e00640 1 -- 172.21.15.49:0/2850671935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12a40b9390 msgr2=0x7f12a41f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.805+0000 7f12a3e00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12a40b9390 0x7f12a41f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.805+0000 7f12a3e00640 1 -- 172.21.15.49:0/2850671935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f128c0051c0 msgr2=0x7f12a4160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.805+0000 7f12a3e00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f128c0051c0 0x7f12a4160990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.805+0000 7f12a3e00640 1 -- 172.21.15.49:0/2850671935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a415fc10 con 0x7f12a414aed0 2024-04-15T15:21:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.805+0000 7f12a3e00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12a414aed0 0x7f12a40b8e50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f129000e4b0 tx=0x7f129000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.805+0000 7f12a0c00640 1 -- 172.21.15.49:0/2850671935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12900052e0 con 0x7f12a414aed0 2024-04-15T15:21:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.806+0000 7f12a0c00640 1 -- 172.21.15.49:0/2850671935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1290018070 con 0x7f12a414aed0 2024-04-15T15:21:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.806+0000 7f12a0c00640 1 -- 172.21.15.49:0/2850671935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290003830 con 0x7f12a414aed0 2024-04-15T15:21:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.806+0000 7f12a9e00640 1 -- 172.21.15.49:0/2850671935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a41f4f50 con 0x7f12a414aed0 2024-04-15T15:21:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.806+0000 7f12a9e00640 1 -- 172.21.15.49:0/2850671935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a4160190 con 0x7f12a414aed0 2024-04-15T15:21:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.807+0000 7f12a9e00640 1 -- 172.21.15.49:0/2850671935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a40755c0 con 0x7f12a414aed0 2024-04-15T15:21:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.808+0000 7f12a0c00640 1 -- 172.21.15.49:0/2850671935 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f12900039d0 con 0x7f12a414aed0 2024-04-15T15:21:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.809+0000 7f12a0c00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f128406c9b0 0x7f128406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.809+0000 7f12a0c00640 1 -- 172.21.15.49:0/2850671935 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f129008b660 con 0x7f12a414aed0 2024-04-15T15:21:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.809+0000 7f12a8a00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f128406c9b0 0x7f128406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.810+0000 7f12a8a00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f128406c9b0 0x7f128406ee70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1298001050 tx=0x7f1298009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:02.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:02.813+0000 7f12a0c00640 1 -- 172.21.15.49:0/2850671935 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1290059c40 con 0x7f12a414aed0 2024-04-15T15:21:03.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.021+0000 7f12a9e00640 1 -- 172.21.15.49:0/2850671935 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f12a40b9770 con 0x7f128406c9b0 2024-04-15T15:21:03.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.022+0000 7f12a0c00640 1 -- 172.21.15.49:0/2850671935 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f12a40b9770 con 0x7f128406c9b0 2024-04-15T15:21:03.024 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 2 to deep-scrub 2024-04-15T15:21:03.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.024+0000 7f12a9e00640 1 -- 172.21.15.49:0/2850671935 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f128406c9b0 msgr2=0x7f128406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.024+0000 7f12a9e00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f128406c9b0 0x7f128406ee70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1298001050 tx=0x7f1298009040 comp rx=0 tx=0).stop 2024-04-15T15:21:03.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.024+0000 7f12a9e00640 1 -- 172.21.15.49:0/2850671935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12a414aed0 msgr2=0x7f12a40b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.024+0000 7f12a9e00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12a414aed0 0x7f12a40b8e50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f129000e4b0 tx=0x7f129000e970 comp rx=0 tx=0).stop 2024-04-15T15:21:03.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.025+0000 7f12a9e00640 1 -- 172.21.15.49:0/2850671935 shutdown_connections 2024-04-15T15:21:03.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.025+0000 7f12a9e00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f128406c9b0 0x7f128406ee70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.025+0000 7f12a9e00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12a40b9390 0x7f12a41f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.025+0000 7f12a9e00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12a414aed0 0x7f12a40b8e50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.025+0000 7f12a9e00640 1 --2- 172.21.15.49:0/2850671935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f128c0051c0 0x7f12a4160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.025+0000 7f12a9e00640 1 -- 172.21.15.49:0/2850671935 >> 172.21.15.49:0/2850671935 conn(0x7f12a40d18d0 msgr2=0x7f12a40b7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.025+0000 7f12a9e00640 1 -- 172.21.15.49:0/2850671935 shutdown_connections 2024-04-15T15:21:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.025+0000 7f12a9e00640 1 -- 172.21.15.49:0/2850671935 wait complete. 2024-04-15T15:21:03.046 INFO:tasks.ceph:Scrubbing osd.3 2024-04-15T15:21:03.047 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:03.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.267+0000 7f81afe00640 1 Processor -- start 2024-04-15T15:21:03.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.267+0000 7f81afe00640 1 -- start start 2024-04-15T15:21:03.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.269+0000 7f81afe00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f81b00cec60 0x7f81b0071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.269+0000 7f81afe00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f81b015d420 0x7f81b006f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.269+0000 7f81afe00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b013fc70 con 0x7f81b0072450 2024-04-15T15:21:03.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.269+0000 7f81afe00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f81b00728c0 con 0x7f81b00cec60 2024-04-15T15:21:03.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.269+0000 7f81afe00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f81b013f9e0 con 0x7f81b015d420 2024-04-15T15:21:03.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.270+0000 7f81aea00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f81b00cec60 0x7f81b0071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.270+0000 7f81aea00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f81b00cec60 0x7f81b0071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43748/0 (socket says 172.21.15.49:43748) 2024-04-15T15:21:03.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.270+0000 7f81aea00640 1 -- 172.21.15.49:0/2502374385 learned_addr learned my addr 172.21.15.49:0/2502374385 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:03.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.270+0000 7f81af400640 1 --2- 172.21.15.49:0/2502374385 >> v2:172.21.15.107:3300/0 conn(0x7f81b015d420 0x7f81b006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.270+0000 7f81aea00640 1 -- 172.21.15.49:0/2502374385 >> v2:172.21.15.107:3300/0 conn(0x7f81b015d420 msgr2=0x7f81b006f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:03.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.270+0000 7f81aea00640 1 --2- 172.21.15.49:0/2502374385 >> v2:172.21.15.107:3300/0 conn(0x7f81b015d420 0x7f81b006f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.270+0000 7f81aea00640 1 -- 172.21.15.49:0/2502374385 >> v1:172.21.15.107:6789/0 conn(0x7f81b0072450 legacy=0x7f81b015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:03.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.271+0000 7f81aea00640 1 -- 172.21.15.49:0/2502374385 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b0141a00 con 0x7f81b00cec60 2024-04-15T15:21:03.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.271+0000 7f81af400640 1 --2- 172.21.15.49:0/2502374385 >> v2:172.21.15.107:3300/0 conn(0x7f81b015d420 0x7f81b006f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:03.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.272+0000 7f81aea00640 1 --2- 172.21.15.49:0/2502374385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b00cec60 0x7f81b0071f10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009870 tx=0x7f81a002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce251daa94f1996 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.272+0000 7f81ad600640 1 -- 172.21.15.49:0/2502374385 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a002f850 con 0x7f81b00cec60 2024-04-15T15:21:03.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.272+0000 7f81ad600640 1 -- 172.21.15.49:0/2502374385 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a002f9f0 con 0x7f81b00cec60 2024-04-15T15:21:03.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.273+0000 7f81ad600640 1 -- 172.21.15.49:0/2502374385 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a002fdc0 con 0x7f81b00cec60 2024-04-15T15:21:03.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.273+0000 7f81afe00640 1 -- 172.21.15.49:0/2502374385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b00cec60 msgr2=0x7f81b0071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.273+0000 7f81afe00640 1 --2- 172.21.15.49:0/2502374385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b00cec60 0x7f81b0071f10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009870 tx=0x7f81a002ba30 comp rx=0 tx=0).stop 2024-04-15T15:21:03.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.273+0000 7f81afe00640 1 -- 172.21.15.49:0/2502374385 shutdown_connections 2024-04-15T15:21:03.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.273+0000 7f81afe00640 1 --2- 172.21.15.49:0/2502374385 >> v2:172.21.15.107:3300/0 conn(0x7f81b015d420 0x7f81b006f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.273+0000 7f81afe00640 1 --2- 172.21.15.49:0/2502374385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b00cec60 0x7f81b0071f10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.273+0000 7f81afe00640 1 -- 172.21.15.49:0/2502374385 >> 172.21.15.49:0/2502374385 conn(0x7f81b00b6170 msgr2=0x7f81b006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:03.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.273+0000 7f81afe00640 1 -- 172.21.15.49:0/2502374385 shutdown_connections 2024-04-15T15:21:03.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.273+0000 7f81afe00640 1 -- 172.21.15.49:0/2502374385 wait complete. 2024-04-15T15:21:03.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.275+0000 7f81afe00640 1 Processor -- start 2024-04-15T15:21:03.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.275+0000 7f81afe00640 1 -- start start 2024-04-15T15:21:03.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.276+0000 7f81afe00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81b0072450 0x7f81b01f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.276+0000 7f81afe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b00cec60 0x7f81b0159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.276+0000 7f81afe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b015d420 0x7f81b01589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.276+0000 7f81afe00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f81b01401f0 con 0x7f81b00cec60 2024-04-15T15:21:03.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.276+0000 7f81afe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f81b013ff10 con 0x7f81b015d420 2024-04-15T15:21:03.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.276+0000 7f81afe00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f81b01404d0 con 0x7f81b0072450 2024-04-15T15:21:03.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81ae000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b00cec60 0x7f81b0159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81ae000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b00cec60 0x7f81b0159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43750/0 (socket says 172.21.15.49:43750) 2024-04-15T15:21:03.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81ae000640 1 -- 172.21.15.49:0/4121692170 learned_addr learned my addr 172.21.15.49:0/4121692170 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:03.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81aea00640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81b0072450 0x7f81b01f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81af400640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b015d420 0x7f81b01589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81ae000640 1 -- 172.21.15.49:0/4121692170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81b0072450 msgr2=0x7f81b01f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81ae000640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f81b0072450 0x7f81b01f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81ae000640 1 -- 172.21.15.49:0/4121692170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b015d420 msgr2=0x7f81b01589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81ae000640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b015d420 0x7f81b01589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81ae000640 1 -- 172.21.15.49:0/4121692170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b0157c30 con 0x7f81b00cec60 2024-04-15T15:21:03.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.277+0000 7f81af400640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b015d420 0x7f81b01589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:03.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.278+0000 7f81ae000640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b00cec60 0x7f81b0159dd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f819c00c400 tx=0x7f819c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.278+0000 7f81a6a00640 1 -- 172.21.15.49:0/4121692170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c0052d0 con 0x7f81b00cec60 2024-04-15T15:21:03.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.278+0000 7f81a6a00640 1 -- 172.21.15.49:0/4121692170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819c005490 con 0x7f81b00cec60 2024-04-15T15:21:03.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.278+0000 7f81a6a00640 1 -- 172.21.15.49:0/4121692170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c007260 con 0x7f81b00cec60 2024-04-15T15:21:03.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.278+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b0157f10 con 0x7f81b00cec60 2024-04-15T15:21:03.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.278+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01f4bf0 con 0x7f81b00cec60 2024-04-15T15:21:03.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.279+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f81b01579e0 con 0x7f81b00cec60 2024-04-15T15:21:03.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.280+0000 7f81a6a00640 1 -- 172.21.15.49:0/4121692170 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f819c007400 con 0x7f81b00cec60 2024-04-15T15:21:03.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.281+0000 7f81a6a00640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f819006c980 0x7f819006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.281+0000 7f81a6a00640 1 -- 172.21.15.49:0/4121692170 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f819c08b5e0 con 0x7f81b00cec60 2024-04-15T15:21:03.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.282+0000 7f81aea00640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f819006c980 0x7f819006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.282+0000 7f81a6a00640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] conn(0x7f81900728e0 0x7f8190074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.282+0000 7f81a6a00640 1 -- 172.21.15.49:0/4121692170 --> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f819c010e50 con 0x7f81900728e0 2024-04-15T15:21:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.282+0000 7f81a6a00640 1 -- 172.21.15.49:0/4121692170 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f819c08bac0 con 0x7f81b00cec60 2024-04-15T15:21:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.282+0000 7f81af400640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] conn(0x7f81900728e0 0x7f8190074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.283+0000 7f81aea00640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f819006c980 0x7f819006ee40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f81a0036000 tx=0x7f81a0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.283+0000 7f81af400640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] conn(0x7f81900728e0 0x7f8190074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.284+0000 7f81a6a00640 1 -- 172.21.15.49:0/4121692170 <== osd.3 v2:172.21.15.49:6822/1234122534 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f819c010e50 con 0x7f81900728e0 2024-04-15T15:21:03.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.299+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 --> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f81b0068b40 con 0x7f81900728e0 2024-04-15T15:21:03.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.299+0000 7f81a6a00640 1 -- 172.21.15.49:0/4121692170 <== osd.3 v2:172.21.15.49:6822/1234122534 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f81b0068b40 con 0x7f81900728e0 2024-04-15T15:21:03.302 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:21:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:03.302 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:21:03.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] conn(0x7f81900728e0 msgr2=0x7f8190074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81afe00640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:6822/1234122534,v1:172.21.15.49:6824/1234122534] conn(0x7f81900728e0 0x7f8190074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f819006c980 msgr2=0x7f819006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81afe00640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f819006c980 0x7f819006ee40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f81a0036000 tx=0x7f81a0008030 comp rx=0 tx=0).stop 2024-04-15T15:21:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b00cec60 msgr2=0x7f81b0159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81afe00640 1 --2- 172.21.15.49:0/4121692170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b00cec60 0x7f81b0159dd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f819c00c400 tx=0x7f819c00c8c0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81af400640 1 -- 172.21.15.49:0/4121692170 reap_dead start 2024-04-15T15:21:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 shutdown_connections 2024-04-15T15:21:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 >> 172.21.15.49:0/4121692170 conn(0x7f81b00b6170 msgr2=0x7f81b015f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 shutdown_connections 2024-04-15T15:21:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.300+0000 7f81afe00640 1 -- 172.21.15.49:0/4121692170 wait complete. 2024-04-15T15:21:03.322 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-15T15:21:03.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.552+0000 7fe1e8400640 1 Processor -- start 2024-04-15T15:21:03.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.552+0000 7fe1e8400640 1 -- start start 2024-04-15T15:21:03.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.553+0000 7fe1e8400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe1e00cf690 0x7fe1e015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e8400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe1e015c130 0x7fe1e006f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e8400640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e00cfd50 con 0x7fe1e00afff0 2024-04-15T15:21:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e8400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e00b3f30 con 0x7fe1e00cf690 2024-04-15T15:21:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e8400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e00b4210 con 0x7fe1e015c130 2024-04-15T15:21:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e5000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe1e00cf690 0x7fe1e015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e5000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe1e00cf690 0x7fe1e015bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43760/0 (socket says 172.21.15.49:43760) 2024-04-15T15:21:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e5000640 1 -- 172.21.15.49:0/68338464 learned_addr learned my addr 172.21.15.49:0/68338464 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:03.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e6400640 1 --2- 172.21.15.49:0/68338464 >> v2:172.21.15.107:3300/0 conn(0x7fe1e015c130 0x7fe1e006f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e5000640 1 -- 172.21.15.49:0/68338464 >> v2:172.21.15.107:3300/0 conn(0x7fe1e015c130 msgr2=0x7fe1e006f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:03.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e5000640 1 --2- 172.21.15.49:0/68338464 >> v2:172.21.15.107:3300/0 conn(0x7fe1e015c130 0x7fe1e006f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e5000640 1 -- 172.21.15.49:0/68338464 >> v1:172.21.15.107:6789/0 conn(0x7fe1e00afff0 legacy=0x7fe1e0144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e5000640 1 -- 172.21.15.49:0/68338464 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e00d0030 con 0x7fe1e00cf690 2024-04-15T15:21:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.554+0000 7fe1e6400640 1 --2- 172.21.15.49:0/68338464 >> v2:172.21.15.107:3300/0 conn(0x7fe1e015c130 0x7fe1e006f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.556+0000 7fe1e5000640 1 --2- 172.21.15.49:0/68338464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00cf690 0x7fe1e015bbf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00cf400 tx=0x7fe1cc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6c8726f489bfc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.556+0000 7fe1dfe00640 1 -- 172.21.15.49:0/68338464 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc02e070 con 0x7fe1e00cf690 2024-04-15T15:21:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.556+0000 7fe1dfe00640 1 -- 172.21.15.49:0/68338464 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1cc037070 con 0x7fe1e00cf690 2024-04-15T15:21:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.556+0000 7fe1dfe00640 1 -- 172.21.15.49:0/68338464 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc031a00 con 0x7fe1e00cf690 2024-04-15T15:21:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.557+0000 7fe1e8400640 1 -- 172.21.15.49:0/68338464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00cf690 msgr2=0x7fe1e015bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.557+0000 7fe1e8400640 1 --2- 172.21.15.49:0/68338464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00cf690 0x7fe1e015bbf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00cf400 tx=0x7fe1cc02b880 comp rx=0 tx=0).stop 2024-04-15T15:21:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.557+0000 7fe1e8400640 1 -- 172.21.15.49:0/68338464 shutdown_connections 2024-04-15T15:21:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.557+0000 7fe1e8400640 1 --2- 172.21.15.49:0/68338464 >> v2:172.21.15.107:3300/0 conn(0x7fe1e015c130 0x7fe1e006f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.557+0000 7fe1e8400640 1 --2- 172.21.15.49:0/68338464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00cf690 0x7fe1e015bbf0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.557+0000 7fe1e8400640 1 -- 172.21.15.49:0/68338464 >> 172.21.15.49:0/68338464 conn(0x7fe1e00b2460 msgr2=0x7fe1e00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.557+0000 7fe1e8400640 1 -- 172.21.15.49:0/68338464 shutdown_connections 2024-04-15T15:21:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.557+0000 7fe1e8400640 1 -- 172.21.15.49:0/68338464 wait complete. 2024-04-15T15:21:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.559+0000 7fe1e8400640 1 Processor -- start 2024-04-15T15:21:03.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.559+0000 7fe1e8400640 1 -- start start 2024-04-15T15:21:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.560+0000 7fe1e8400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00afff0 0x7fe1e00ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.560+0000 7fe1e8400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1e00cf690 0x7fe1e015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.560+0000 7fe1e8400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e015c130 0x7fe1e0136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.560+0000 7fe1e8400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e006f750 con 0x7fe1e00afff0 2024-04-15T15:21:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.560+0000 7fe1e8400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e00cfaf0 con 0x7fe1e015c130 2024-04-15T15:21:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.560+0000 7fe1e8400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e006f8d0 con 0x7fe1e00cf690 2024-04-15T15:21:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.560+0000 7fe1e5a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00afff0 0x7fe1e00ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.560+0000 7fe1e5a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00afff0 0x7fe1e00ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43772/0 (socket says 172.21.15.49:43772) 2024-04-15T15:21:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.560+0000 7fe1e5a00640 1 -- 172.21.15.49:0/2169892196 learned_addr learned my addr 172.21.15.49:0/2169892196 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:03.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1e5000640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1e00cf690 0x7fe1e015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1e6400640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e015c130 0x7fe1e0136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1e5a00640 1 -- 172.21.15.49:0/2169892196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1e00cf690 msgr2=0x7fe1e015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1e5a00640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1e00cf690 0x7fe1e015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1e5a00640 1 -- 172.21.15.49:0/2169892196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e015c130 msgr2=0x7fe1e0136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1e5a00640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e015c130 0x7fe1e0136c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1e5a00640 1 -- 172.21.15.49:0/2169892196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e0135b70 con 0x7fe1e00afff0 2024-04-15T15:21:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1e5000640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1e00cf690 0x7fe1e015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1e5a00640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00afff0 0x7fe1e00ab4b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400c410 tx=0x7fe1d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1dd600640 1 -- 172.21.15.49:0/2169892196 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4017070 con 0x7fe1e00afff0 2024-04-15T15:21:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1dd600640 1 -- 172.21.15.49:0/2169892196 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d40035f0 con 0x7fe1e00afff0 2024-04-15T15:21:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.561+0000 7fe1dd600640 1 -- 172.21.15.49:0/2169892196 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4003a20 con 0x7fe1e00afff0 2024-04-15T15:21:03.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.562+0000 7fe1e8400640 1 -- 172.21.15.49:0/2169892196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e0135e50 con 0x7fe1e00afff0 2024-04-15T15:21:03.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.562+0000 7fe1e8400640 1 -- 172.21.15.49:0/2169892196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e0136410 con 0x7fe1e00afff0 2024-04-15T15:21:03.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.563+0000 7fe1e8400640 1 -- 172.21.15.49:0/2169892196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e0068c80 con 0x7fe1e00afff0 2024-04-15T15:21:03.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.564+0000 7fe1dd600640 1 -- 172.21.15.49:0/2169892196 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe1d4003bc0 con 0x7fe1e00afff0 2024-04-15T15:21:03.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.565+0000 7fe1dd600640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe1c006c980 0x7fe1c006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:03.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.565+0000 7fe1dd600640 1 -- 172.21.15.49:0/2169892196 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fe1d408ad30 con 0x7fe1e00afff0 2024-04-15T15:21:03.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.565+0000 7fe1e5000640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe1c006c980 0x7fe1c006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:03.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.566+0000 7fe1e5000640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe1c006c980 0x7fe1c006ee40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc036000 tx=0x7fe1cc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:03.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.570+0000 7fe1dd600640 1 -- 172.21.15.49:0/2169892196 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1d4059310 con 0x7fe1e00afff0 2024-04-15T15:21:03.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.777+0000 7fe1e8400640 1 -- 172.21.15.49:0/2169892196 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe1e0144460 con 0x7fe1c006c980 2024-04-15T15:21:03.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.777+0000 7fe1dd600640 1 -- 172.21.15.49:0/2169892196 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe1e0144460 con 0x7fe1c006c980 2024-04-15T15:21:03.780 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 3 to deep-scrub 2024-04-15T15:21:03.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.780+0000 7fe1e8400640 1 -- 172.21.15.49:0/2169892196 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe1c006c980 msgr2=0x7fe1c006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.780+0000 7fe1e8400640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe1c006c980 0x7fe1c006ee40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc036000 tx=0x7fe1cc008030 comp rx=0 tx=0).stop 2024-04-15T15:21:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.780+0000 7fe1e8400640 1 -- 172.21.15.49:0/2169892196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00afff0 msgr2=0x7fe1e00ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.780+0000 7fe1e8400640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00afff0 0x7fe1e00ab4b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400c410 tx=0x7fe1d400c8d0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.780+0000 7fe1e8400640 1 -- 172.21.15.49:0/2169892196 shutdown_connections 2024-04-15T15:21:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.780+0000 7fe1e8400640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fe1c006c980 0x7fe1c006ee40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.780+0000 7fe1e8400640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1e00cf690 0x7fe1e015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.780+0000 7fe1e8400640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e015c130 0x7fe1e0136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.780+0000 7fe1e8400640 1 --2- 172.21.15.49:0/2169892196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00afff0 0x7fe1e00ab4b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.780+0000 7fe1e8400640 1 -- 172.21.15.49:0/2169892196 >> 172.21.15.49:0/2169892196 conn(0x7fe1e00b2460 msgr2=0x7fe1e00b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.781+0000 7fe1e8400640 1 -- 172.21.15.49:0/2169892196 shutdown_connections 2024-04-15T15:21:03.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:03.781+0000 7fe1e8400640 1 -- 172.21.15.49:0/2169892196 wait complete. 2024-04-15T15:21:03.804 INFO:tasks.ceph:Scrubbing osd.4 2024-04-15T15:21:03.805 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:04.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.027+0000 7f5958c00640 1 Processor -- start 2024-04-15T15:21:04.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.027+0000 7f5958c00640 1 -- start start 2024-04-15T15:21:04.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.029+0000 7f5958c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f59541462f0 0x7f5954072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.029+0000 7f5958c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5954072690 0x7f59540cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.029+0000 7f5958c00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59540ce000 con 0x7f59540cd680 2024-04-15T15:21:04.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.029+0000 7f5958c00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f59540a9640 con 0x7f59541462f0 2024-04-15T15:21:04.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.029+0000 7f5958c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f5954072af0 con 0x7f5954072690 2024-04-15T15:21:04.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.029+0000 7f5951e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f59541462f0 0x7f5954072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.029+0000 7f5951e00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f59541462f0 0x7f5954072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58606/0 (socket says 172.21.15.49:58606) 2024-04-15T15:21:04.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.029+0000 7f5951e00640 1 -- 172.21.15.49:0/3216143498 learned_addr learned my addr 172.21.15.49:0/3216143498 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:04.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.030+0000 7f5951400640 1 --2- 172.21.15.49:0/3216143498 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5954072690 0x7f59540cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.030+0000 7f5951e00640 1 -- 172.21.15.49:0/3216143498 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5954072690 msgr2=0x7f59540cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:04.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.030+0000 7f5951e00640 1 --2- 172.21.15.49:0/3216143498 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5954072690 0x7f59540cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.030+0000 7f5951e00640 1 -- 172.21.15.49:0/3216143498 >> v1:172.21.15.107:6789/0 conn(0x7f59540cd680 legacy=0x7f595406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:04.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.030+0000 7f5951e00640 1 -- 172.21.15.49:0/3216143498 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59540cdd20 con 0x7f59541462f0 2024-04-15T15:21:04.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.030+0000 7f5951400640 1 --2- 172.21.15.49:0/3216143498 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5954072690 0x7f59540cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:04.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.031+0000 7f5951e00640 1 --2- 172.21.15.49:0/3216143498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59541462f0 0x7f5954072150 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5944009870 tx=0x7f594402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6af28ab107fe74 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.031+0000 7f5950a00640 1 -- 172.21.15.49:0/3216143498 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944034070 con 0x7f59541462f0 2024-04-15T15:21:04.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.031+0000 7f5950a00640 1 -- 172.21.15.49:0/3216143498 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5944030840 con 0x7f59541462f0 2024-04-15T15:21:04.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.031+0000 7f5950a00640 1 -- 172.21.15.49:0/3216143498 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944030c10 con 0x7f59541462f0 2024-04-15T15:21:04.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.032+0000 7f5958c00640 1 -- 172.21.15.49:0/3216143498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59541462f0 msgr2=0x7f5954072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.032+0000 7f5958c00640 1 --2- 172.21.15.49:0/3216143498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59541462f0 0x7f5954072150 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5944009870 tx=0x7f594402b980 comp rx=0 tx=0).stop 2024-04-15T15:21:04.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.032+0000 7f5958c00640 1 -- 172.21.15.49:0/3216143498 shutdown_connections 2024-04-15T15:21:04.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.032+0000 7f5958c00640 1 --2- 172.21.15.49:0/3216143498 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5954072690 0x7f59540cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.032+0000 7f5958c00640 1 --2- 172.21.15.49:0/3216143498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59541462f0 0x7f5954072150 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.032+0000 7f5958c00640 1 -- 172.21.15.49:0/3216143498 >> 172.21.15.49:0/3216143498 conn(0x7f59540b2460 msgr2=0x7f59540b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:04.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.032+0000 7f5958c00640 1 -- 172.21.15.49:0/3216143498 shutdown_connections 2024-04-15T15:21:04.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.032+0000 7f5958c00640 1 -- 172.21.15.49:0/3216143498 wait complete. 2024-04-15T15:21:04.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.035+0000 7f5958c00640 1 Processor -- start 2024-04-15T15:21:04.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.035+0000 7f5958c00640 1 -- start start 2024-04-15T15:21:04.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.036+0000 7f5958c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5954072690 0x7f5954156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.036+0000 7f5958c00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59540cd680 0x7f5954159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.036+0000 7f5958c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59541462f0 0x7f5954142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.036+0000 7f5958c00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f595415d4d0 con 0x7f59541462f0 2024-04-15T15:21:04.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.036+0000 7f5958c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f595415d1f0 con 0x7f5954072690 2024-04-15T15:21:04.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.036+0000 7f5958c00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f595415ea00 con 0x7f59540cd680 2024-04-15T15:21:04.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.036+0000 7f5952800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59541462f0 0x7f5954142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.036+0000 7f5952800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59541462f0 0x7f5954142200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58620/0 (socket says 172.21.15.49:58620) 2024-04-15T15:21:04.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.036+0000 7f5952800640 1 -- 172.21.15.49:0/1642685919 learned_addr learned my addr 172.21.15.49:0/1642685919 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:04.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.037+0000 7f5951e00640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5954072690 0x7f5954156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.037+0000 7f5951400640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59540cd680 0x7f5954159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.037+0000 7f5952800640 1 -- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59540cd680 msgr2=0x7f5954159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.037+0000 7f5952800640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59540cd680 0x7f5954159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.037+0000 7f5952800640 1 -- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5954072690 msgr2=0x7f5954156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.037+0000 7f5952800640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5954072690 0x7f5954156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.037+0000 7f5952800640 1 -- 172.21.15.49:0/1642685919 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595413d3a0 con 0x7f59541462f0 2024-04-15T15:21:04.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.037+0000 7f5952800640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59541462f0 0x7f5954142200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f594c00c780 tx=0x7f594c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.038+0000 7f594a000640 1 -- 172.21.15.49:0/1642685919 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c004b60 con 0x7f59541462f0 2024-04-15T15:21:04.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.038+0000 7f594a000640 1 -- 172.21.15.49:0/1642685919 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594c004d00 con 0x7f59541462f0 2024-04-15T15:21:04.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.038+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5954157d20 con 0x7f59541462f0 2024-04-15T15:21:04.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.038+0000 7f594a000640 1 -- 172.21.15.49:0/1642685919 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c00aa90 con 0x7f59541462f0 2024-04-15T15:21:04.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.038+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59541582e0 con 0x7f59541462f0 2024-04-15T15:21:04.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.039+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5954068db0 con 0x7f59541462f0 2024-04-15T15:21:04.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.040+0000 7f594a000640 1 -- 172.21.15.49:0/1642685919 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f594c00ac30 con 0x7f59541462f0 2024-04-15T15:21:04.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.041+0000 7f594a000640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f593406c980 0x7f593406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.041+0000 7f594a000640 1 -- 172.21.15.49:0/1642685919 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f594c08c400 con 0x7f59541462f0 2024-04-15T15:21:04.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.042+0000 7f594a000640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] conn(0x7f59340728e0 0x7f5934074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.042+0000 7f5951e00640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f593406c980 0x7f593406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.042+0000 7f594a000640 1 -- 172.21.15.49:0/1642685919 --> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f594c025e50 con 0x7f59340728e0 2024-04-15T15:21:04.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.042+0000 7f594a000640 1 -- 172.21.15.49:0/1642685919 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f594c08c7b0 con 0x7f59541462f0 2024-04-15T15:21:04.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.042+0000 7f5951400640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] conn(0x7f59340728e0 0x7f5934074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.042+0000 7f5951e00640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f593406c980 0x7f593406ee40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5944036000 tx=0x7f594403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.043+0000 7f5951400640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] conn(0x7f59340728e0 0x7f5934074d00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.046+0000 7f594a000640 1 -- 172.21.15.49:0/1642685919 <== osd.4 v2:172.21.15.107:6800/3185658252 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f594c025e50 con 0x7f59340728e0 2024-04-15T15:21:04.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.060+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 --> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f59540cfd50 con 0x7f59340728e0 2024-04-15T15:21:04.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.061+0000 7f594a000640 1 -- 172.21.15.49:0/1642685919 <== osd.4 v2:172.21.15.107:6800/3185658252 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f59540cfd50 con 0x7f59340728e0 2024-04-15T15:21:04.064 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:21:04.064 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:04.064 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:21:04.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.062+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] conn(0x7f59340728e0 msgr2=0x7f5934074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.062+0000 7f5958c00640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.107:6800/3185658252,v1:172.21.15.107:6801/3185658252] conn(0x7f59340728e0 0x7f5934074d00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.062+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f593406c980 msgr2=0x7f593406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.062+0000 7f5958c00640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f593406c980 0x7f593406ee40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5944036000 tx=0x7f594403b040 comp rx=0 tx=0).stop 2024-04-15T15:21:04.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.062+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59541462f0 msgr2=0x7f5954142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.062+0000 7f5958c00640 1 --2- 172.21.15.49:0/1642685919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59541462f0 0x7f5954142200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f594c00c780 tx=0x7f594c00cc40 comp rx=0 tx=0).stop 2024-04-15T15:21:04.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.062+0000 7f5952800640 1 -- 172.21.15.49:0/1642685919 reap_dead start 2024-04-15T15:21:04.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.063+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 shutdown_connections 2024-04-15T15:21:04.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.063+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 >> 172.21.15.49:0/1642685919 conn(0x7f59540b2460 msgr2=0x7f595415ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:04.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.063+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 shutdown_connections 2024-04-15T15:21:04.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.063+0000 7f5958c00640 1 -- 172.21.15.49:0/1642685919 wait complete. 2024-04-15T15:21:04.085 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-15T15:21:04.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.315+0000 7f6073000640 1 Processor -- start 2024-04-15T15:21:04.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.315+0000 7f6073000640 1 -- start start 2024-04-15T15:21:04.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.316+0000 7f6073000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f606c148110 0x7f606c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.316+0000 7f6073000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c15f4e0 0x7f606c0aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.316+0000 7f6073000640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f606c0b43b0 0x7f606c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.316+0000 7f6073000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f606c15ff20 con 0x7f606c148110 2024-04-15T15:21:04.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f6073000640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f606c15f960 con 0x7f606c0b43b0 2024-04-15T15:21:04.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f6073000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f606c15fc40 con 0x7f606c15f4e0 2024-04-15T15:21:04.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f606be00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f606c148110 0x7f606c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f606be00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f606c148110 0x7f606c15efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58624/0 (socket says 172.21.15.49:58624) 2024-04-15T15:21:04.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f606be00640 1 -- 172.21.15.49:0/2624299223 learned_addr learned my addr 172.21.15.49:0/2624299223 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:04.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f6063400640 1 --2- 172.21.15.49:0/2624299223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c15f4e0 0x7f606c0aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f6071000640 1 --2- 172.21.15.49:0/2624299223 >> v2:172.21.15.107:3300/0 conn(0x7f606c0b43b0 0x7f606c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f606be00640 1 -- 172.21.15.49:0/2624299223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c15f4e0 msgr2=0x7f606c0aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:04.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f606be00640 1 --2- 172.21.15.49:0/2624299223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c15f4e0 0x7f606c0aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f606be00640 1 -- 172.21.15.49:0/2624299223 >> v2:172.21.15.107:3300/0 conn(0x7f606c0b43b0 msgr2=0x7f606c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:04.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f606be00640 1 --2- 172.21.15.49:0/2624299223 >> v2:172.21.15.107:3300/0 conn(0x7f606c0b43b0 0x7f606c072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f606be00640 1 -- 172.21.15.49:0/2624299223 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c0b4b20 con 0x7f606c148110 2024-04-15T15:21:04.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f6063400640 1 --2- 172.21.15.49:0/2624299223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c15f4e0 0x7f606c0aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:04.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.317+0000 7f6071000640 1 --2- 172.21.15.49:0/2624299223 >> v2:172.21.15.107:3300/0 conn(0x7f606c0b43b0 0x7f606c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:04.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.319+0000 7f606be00640 1 --2- 172.21.15.49:0/2624299223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c148110 0x7f606c15efa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f606c0ae230 tx=0x7f605c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5926c48e9d1a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.319+0000 7f606b400640 1 -- 172.21.15.49:0/2624299223 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c035020 con 0x7f606c148110 2024-04-15T15:21:04.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.319+0000 7f606b400640 1 -- 172.21.15.49:0/2624299223 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605c031880 con 0x7f606c148110 2024-04-15T15:21:04.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.319+0000 7f606b400640 1 -- 172.21.15.49:0/2624299223 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c031c50 con 0x7f606c148110 2024-04-15T15:21:04.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.320+0000 7f6073000640 1 -- 172.21.15.49:0/2624299223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c148110 msgr2=0x7f606c15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.320+0000 7f6073000640 1 --2- 172.21.15.49:0/2624299223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c148110 0x7f606c15efa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f606c0ae230 tx=0x7f605c02b920 comp rx=0 tx=0).stop 2024-04-15T15:21:04.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.320+0000 7f6073000640 1 -- 172.21.15.49:0/2624299223 shutdown_connections 2024-04-15T15:21:04.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.320+0000 7f6073000640 1 --2- 172.21.15.49:0/2624299223 >> v2:172.21.15.107:3300/0 conn(0x7f606c0b43b0 0x7f606c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.320+0000 7f6073000640 1 --2- 172.21.15.49:0/2624299223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c15f4e0 0x7f606c0aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.320+0000 7f6073000640 1 --2- 172.21.15.49:0/2624299223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c148110 0x7f606c15efa0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.320+0000 7f6073000640 1 -- 172.21.15.49:0/2624299223 >> 172.21.15.49:0/2624299223 conn(0x7f606c0b2460 msgr2=0x7f606c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:04.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.320+0000 7f6073000640 1 -- 172.21.15.49:0/2624299223 shutdown_connections 2024-04-15T15:21:04.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.320+0000 7f6073000640 1 -- 172.21.15.49:0/2624299223 wait complete. 2024-04-15T15:21:04.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.322+0000 7f6073000640 1 Processor -- start 2024-04-15T15:21:04.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.322+0000 7f6073000640 1 -- start start 2024-04-15T15:21:04.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.323+0000 7f6073000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c0b43b0 0x7f606c141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.323+0000 7f6073000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c148110 0x7f606c1f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.323+0000 7f6073000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f606c15f4e0 0x7f606c12f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.323+0000 7f6073000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f606c148670 con 0x7f606c0b43b0 2024-04-15T15:21:04.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.323+0000 7f6073000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1484f0 con 0x7f606c15f4e0 2024-04-15T15:21:04.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.323+0000 7f6073000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f606c0af1f0 con 0x7f606c148110 2024-04-15T15:21:04.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.323+0000 7f606be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c0b43b0 0x7f606c141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.323+0000 7f606be00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c0b43b0 0x7f606c141220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58638/0 (socket says 172.21.15.49:58638) 2024-04-15T15:21:04.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.323+0000 7f606be00640 1 -- 172.21.15.49:0/1282070335 learned_addr learned my addr 172.21.15.49:0/1282070335 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:04.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.324+0000 7f6063400640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c148110 0x7f606c1f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.324+0000 7f606be00640 1 -- 172.21.15.49:0/1282070335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c148110 msgr2=0x7f606c1f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.324+0000 7f606be00640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c148110 0x7f606c1f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.324+0000 7f606be00640 1 -- 172.21.15.49:0/1282070335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f606c15f4e0 msgr2=0x7f606c12f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.324+0000 7f606be00640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f606c15f4e0 0x7f606c12f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.324+0000 7f606be00640 1 -- 172.21.15.49:0/1282070335 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c141b20 con 0x7f606c0b43b0 2024-04-15T15:21:04.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.324+0000 7f6063400640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c148110 0x7f606c1f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:04.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.324+0000 7f606be00640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c0b43b0 0x7f606c141220 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f605c000c00 tx=0x7f605c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.325+0000 7f6068c00640 1 -- 172.21.15.49:0/1282070335 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c040070 con 0x7f606c0b43b0 2024-04-15T15:21:04.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.325+0000 7f6068c00640 1 -- 172.21.15.49:0/1282070335 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605c03e070 con 0x7f606c0b43b0 2024-04-15T15:21:04.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.325+0000 7f6068c00640 1 -- 172.21.15.49:0/1282070335 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c035040 con 0x7f606c0b43b0 2024-04-15T15:21:04.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.325+0000 7f6073000640 1 -- 172.21.15.49:0/1282070335 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606c141e00 con 0x7f606c0b43b0 2024-04-15T15:21:04.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.325+0000 7f6073000640 1 -- 172.21.15.49:0/1282070335 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606c1423c0 con 0x7f606c0b43b0 2024-04-15T15:21:04.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.326+0000 7f6073000640 1 -- 172.21.15.49:0/1282070335 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606c068c80 con 0x7f606c0b43b0 2024-04-15T15:21:04.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.327+0000 7f6068c00640 1 -- 172.21.15.49:0/1282070335 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f605c038020 con 0x7f606c0b43b0 2024-04-15T15:21:04.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.328+0000 7f6068c00640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f604806c980 0x7f604806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.328+0000 7f6068c00640 1 -- 172.21.15.49:0/1282070335 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f605c0ada30 con 0x7f606c0b43b0 2024-04-15T15:21:04.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.328+0000 7f6063400640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f604806c980 0x7f604806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.329+0000 7f6063400640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f604806c980 0x7f604806ee40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6054006fd0 tx=0x7f6054008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.332+0000 7f6068c00640 1 -- 172.21.15.49:0/1282070335 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f605c07c0a0 con 0x7f606c0b43b0 2024-04-15T15:21:04.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.536+0000 7f6073000640 1 -- 172.21.15.49:0/1282070335 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f606c0afb00 con 0x7f604806c980 2024-04-15T15:21:04.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.537+0000 7f6068c00640 1 -- 172.21.15.49:0/1282070335 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f606c0afb00 con 0x7f604806c980 2024-04-15T15:21:04.539 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 4 to deep-scrub 2024-04-15T15:21:04.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.539+0000 7f6073000640 1 -- 172.21.15.49:0/1282070335 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f604806c980 msgr2=0x7f604806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.539+0000 7f6073000640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f604806c980 0x7f604806ee40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6054006fd0 tx=0x7f6054008040 comp rx=0 tx=0).stop 2024-04-15T15:21:04.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.539+0000 7f6073000640 1 -- 172.21.15.49:0/1282070335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c0b43b0 msgr2=0x7f606c141220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.539+0000 7f6073000640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c0b43b0 0x7f606c141220 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f605c000c00 tx=0x7f605c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.539+0000 7f6073000640 1 -- 172.21.15.49:0/1282070335 shutdown_connections 2024-04-15T15:21:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.539+0000 7f6073000640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f604806c980 0x7f604806ee40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.539+0000 7f6073000640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f606c148110 0x7f606c1f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.539+0000 7f6073000640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f606c15f4e0 0x7f606c12f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.539+0000 7f6073000640 1 --2- 172.21.15.49:0/1282070335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f606c0b43b0 0x7f606c141220 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.539+0000 7f6073000640 1 -- 172.21.15.49:0/1282070335 >> 172.21.15.49:0/1282070335 conn(0x7f606c0b2460 msgr2=0x7f606c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.540+0000 7f6073000640 1 -- 172.21.15.49:0/1282070335 shutdown_connections 2024-04-15T15:21:04.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.540+0000 7f6073000640 1 -- 172.21.15.49:0/1282070335 wait complete. 2024-04-15T15:21:04.563 INFO:tasks.ceph:Scrubbing osd.5 2024-04-15T15:21:04.563 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:04.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.796+0000 7f747b000640 1 Processor -- start 2024-04-15T15:21:04.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.797+0000 7f747b000640 1 -- start start 2024-04-15T15:21:04.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.798+0000 7f747b000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74740b52f0 0x7f747406f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.798+0000 7f747b000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74740b56d0 0x7f74740b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.798+0000 7f747b000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747415f340 con 0x7f747415edd0 2024-04-15T15:21:04.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.798+0000 7f747b000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f747415f620 con 0x7f74740b52f0 2024-04-15T15:21:04.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.798+0000 7f747b000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f747415f900 con 0x7f74740b56d0 2024-04-15T15:21:04.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.799+0000 7f7479c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74740b52f0 0x7f747406f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.799+0000 7f7479c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74740b52f0 0x7f747406f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58648/0 (socket says 172.21.15.49:58648) 2024-04-15T15:21:04.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.799+0000 7f7479c00640 1 -- 172.21.15.49:0/3134218516 learned_addr learned my addr 172.21.15.49:0/3134218516 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:04.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.799+0000 7f7479200640 1 --2- 172.21.15.49:0/3134218516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74740b56d0 0x7f74740b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.799+0000 7f7479c00640 1 -- 172.21.15.49:0/3134218516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74740b56d0 msgr2=0x7f74740b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:04.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.799+0000 7f7479c00640 1 --2- 172.21.15.49:0/3134218516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74740b56d0 0x7f74740b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.799+0000 7f7479c00640 1 -- 172.21.15.49:0/3134218516 >> v1:172.21.15.107:6789/0 conn(0x7f747415edd0 legacy=0x7f7474071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:04.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.799+0000 7f7479c00640 1 -- 172.21.15.49:0/3134218516 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747415fbe0 con 0x7f74740b52f0 2024-04-15T15:21:04.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.799+0000 7f7479200640 1 --2- 172.21.15.49:0/3134218516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74740b56d0 0x7f74740b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:04.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.801+0000 7f7479c00640 1 --2- 172.21.15.49:0/3134218516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74740b52f0 0x7f747406f760 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7468009870 tx=0x7f746802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b596f55b3d338668 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.801+0000 7f7473e00640 1 -- 172.21.15.49:0/3134218516 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468035020 con 0x7f74740b52f0 2024-04-15T15:21:04.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.801+0000 7f7473e00640 1 -- 172.21.15.49:0/3134218516 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7468031640 con 0x7f74740b52f0 2024-04-15T15:21:04.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.801+0000 7f7473e00640 1 -- 172.21.15.49:0/3134218516 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746802d040 con 0x7f74740b52f0 2024-04-15T15:21:04.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.802+0000 7f747b000640 1 -- 172.21.15.49:0/3134218516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74740b52f0 msgr2=0x7f747406f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.802+0000 7f747b000640 1 --2- 172.21.15.49:0/3134218516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74740b52f0 0x7f747406f760 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7468009870 tx=0x7f746802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.802+0000 7f747b000640 1 -- 172.21.15.49:0/3134218516 shutdown_connections 2024-04-15T15:21:04.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.802+0000 7f747b000640 1 --2- 172.21.15.49:0/3134218516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74740b56d0 0x7f74740b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.802+0000 7f747b000640 1 --2- 172.21.15.49:0/3134218516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74740b52f0 0x7f747406f760 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.802+0000 7f747b000640 1 -- 172.21.15.49:0/3134218516 >> 172.21.15.49:0/3134218516 conn(0x7f74740b6170 msgr2=0x7f747406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:04.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.802+0000 7f747b000640 1 -- 172.21.15.49:0/3134218516 shutdown_connections 2024-04-15T15:21:04.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.802+0000 7f747b000640 1 -- 172.21.15.49:0/3134218516 wait complete. 2024-04-15T15:21:04.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.804+0000 7f747b000640 1 Processor -- start 2024-04-15T15:21:04.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.804+0000 7f747b000640 1 -- start start 2024-04-15T15:21:04.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.805+0000 7f747b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74740b52f0 0x7f7474154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.805+0000 7f747b000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74740b56d0 0x7f7474153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.805+0000 7f747b000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f747415edd0 0x7f7474146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.805+0000 7f747b000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7474072620 con 0x7f74740b56d0 2024-04-15T15:21:04.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.805+0000 7f747b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74740b3b90 con 0x7f74740b52f0 2024-04-15T15:21:04.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.805+0000 7f747b000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f74740727a0 con 0x7f747415edd0 2024-04-15T15:21:04.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.805+0000 7f7479200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74740b56d0 0x7f7474153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.805+0000 7f7479200640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74740b56d0 0x7f7474153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58660/0 (socket says 172.21.15.49:58660) 2024-04-15T15:21:04.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.805+0000 7f7479200640 1 -- 172.21.15.49:0/644589469 learned_addr learned my addr 172.21.15.49:0/644589469 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:04.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f7479c00640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74740b52f0 0x7f7474154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f747a600640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f747415edd0 0x7f7474146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f7479200640 1 -- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f747415edd0 msgr2=0x7f7474146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f7479200640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f747415edd0 0x7f7474146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f7479200640 1 -- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74740b52f0 msgr2=0x7f7474154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f7479200640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74740b52f0 0x7f7474154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f7479200640 1 -- 172.21.15.49:0/644589469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7474153860 con 0x7f74740b56d0 2024-04-15T15:21:04.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f7479c00640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74740b52f0 0x7f7474154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:04.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f747a600640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f747415edd0 0x7f7474146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:04.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f7479200640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74740b56d0 0x7f7474153160 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f746000c7f0 tx=0x7f746000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.806+0000 7f7471600640 1 -- 172.21.15.49:0/644589469 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460015070 con 0x7f74740b56d0 2024-04-15T15:21:04.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.807+0000 7f7471600640 1 -- 172.21.15.49:0/644589469 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7460007a30 con 0x7f74740b56d0 2024-04-15T15:21:04.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.807+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7474153b40 con 0x7f74740b56d0 2024-04-15T15:21:04.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.807+0000 7f7471600640 1 -- 172.21.15.49:0/644589469 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460007e00 con 0x7f74740b56d0 2024-04-15T15:21:04.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.807+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7474154100 con 0x7f74740b56d0 2024-04-15T15:21:04.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.808+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7474068b40 con 0x7f74740b56d0 2024-04-15T15:21:04.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.809+0000 7f7471600640 1 -- 172.21.15.49:0/644589469 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7460018070 con 0x7f74740b56d0 2024-04-15T15:21:04.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.810+0000 7f7471600640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f745406c980 0x7f745406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.810+0000 7f7471600640 1 -- 172.21.15.49:0/644589469 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f746008bb00 con 0x7f74740b56d0 2024-04-15T15:21:04.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.810+0000 7f7471600640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] conn(0x7f74540728e0 0x7f7454074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:04.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.810+0000 7f7471600640 1 -- 172.21.15.49:0/644589469 --> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74540753b0 con 0x7f74540728e0 2024-04-15T15:21:04.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.810+0000 7f7479c00640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f745406c980 0x7f745406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.810+0000 7f7471600640 1 -- 172.21.15.49:0/644589469 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f74600902a0 con 0x7f74740b56d0 2024-04-15T15:21:04.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.811+0000 7f7479c00640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f745406c980 0x7f745406ee40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7468036000 tx=0x7f7468008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.811+0000 7f747a600640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] conn(0x7f74540728e0 0x7f7454074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:04.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.812+0000 7f747a600640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] conn(0x7f74540728e0 0x7f7454074d00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:04.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.814+0000 7f7471600640 1 -- 172.21.15.49:0/644589469 <== osd.5 v2:172.21.15.107:6808/1708671582 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74540753b0 con 0x7f74540728e0 2024-04-15T15:21:04.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.829+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 --> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f74740b0040 con 0x7f74540728e0 2024-04-15T15:21:04.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.831+0000 7f7471600640 1 -- 172.21.15.49:0/644589469 <== osd.5 v2:172.21.15.107:6808/1708671582 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f74740b0040 con 0x7f74540728e0 2024-04-15T15:21:04.834 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:21:04.834 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:04.834 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:21:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.831+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] conn(0x7f74540728e0 msgr2=0x7f7454074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.831+0000 7f747b000640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.107:6808/1708671582,v1:172.21.15.107:6810/1708671582] conn(0x7f74540728e0 0x7f7454074d00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.831+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f745406c980 msgr2=0x7f745406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.831+0000 7f747b000640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f745406c980 0x7f745406ee40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7468036000 tx=0x7f7468008030 comp rx=0 tx=0).stop 2024-04-15T15:21:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.831+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74740b56d0 msgr2=0x7f7474153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.831+0000 7f747b000640 1 --2- 172.21.15.49:0/644589469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74740b56d0 0x7f7474153160 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f746000c7f0 tx=0x7f746000cbf0 comp rx=0 tx=0).stop 2024-04-15T15:21:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.831+0000 7f747a600640 1 -- 172.21.15.49:0/644589469 reap_dead start 2024-04-15T15:21:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.831+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 shutdown_connections 2024-04-15T15:21:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.831+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 >> 172.21.15.49:0/644589469 conn(0x7f74740b6170 msgr2=0x7f74740af930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:04.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.832+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 shutdown_connections 2024-04-15T15:21:04.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:04.832+0000 7f747b000640 1 -- 172.21.15.49:0/644589469 wait complete. 2024-04-15T15:21:04.854 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-15T15:21:05.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.077+0000 7f7995e00640 1 Processor -- start 2024-04-15T15:21:05.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.077+0000 7f7995e00640 1 -- start start 2024-04-15T15:21:05.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.079+0000 7f7995e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f799015b410 0x7f799015c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.079+0000 7f7995e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900cf530 0x7f79900cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.079+0000 7f7995e00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f799015cae0 con 0x7f799015f110 2024-04-15T15:21:05.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.079+0000 7f7995e00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f79900cd360 con 0x7f799015b410 2024-04-15T15:21:05.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.079+0000 7f7995e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f79900cd640 con 0x7f79900cf530 2024-04-15T15:21:05.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.079+0000 7f7995400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f799015f110 0x7f799015f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42598/0 (socket says 172.21.15.49:42598) 2024-04-15T15:21:05.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.079+0000 7f7995400640 1 -- 172.21.15.49:0/914898459 learned_addr learned my addr 172.21.15.49:0/914898459 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:21:05.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798fe00640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900cf530 0x7f79900cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f7994a00640 1 --2- 172.21.15.49:0/914898459 >> v2:172.21.15.107:3300/0 conn(0x7f799015b410 0x7f799015c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1789547669 0 0) 0x7f7988001110 con 0x7f799015f110 2024-04-15T15:21:05.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7974005730 con 0x7f799015f110 2024-04-15T15:21:05.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 571179001 0 0) 0x7f7974005730 con 0x7f799015f110 2024-04-15T15:21:05.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900cf530 msgr2=0x7f79900cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:05.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900cf530 0x7f79900cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 >> v2:172.21.15.107:3300/0 conn(0x7f799015b410 msgr2=0x7f799015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:05.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 --2- 172.21.15.49:0/914898459 >> v2:172.21.15.107:3300/0 conn(0x7f799015b410 0x7f799015c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79900cd920 con 0x7f799015f110 2024-04-15T15:21:05.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2948519169 0 0) 0x7f7988001840 con 0x7f799015f110 2024-04-15T15:21:05.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 >> v1:172.21.15.49:6789/0 conn(0x7f799015f110 legacy=0x7f799015f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7974005bc0 0x7f7974005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900cf530 0x7f7974006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7974006f50 0x7f79740073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f79900cd360 con 0x7f7974006f50 2024-04-15T15:21:05.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79900cd640 con 0x7f7974005bc0 2024-04-15T15:21:05.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.080+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f797400dbd0 con 0x7f79900cf530 2024-04-15T15:21:05.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.081+0000 7f7995400640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7974006f50 0x7f79740073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.081+0000 7f7994a00640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7974005bc0 0x7f7974005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.081+0000 7f798fe00640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900cf530 0x7f7974006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.081+0000 7f7995400640 1 -- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900cf530 msgr2=0x7f7974006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:05.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.081+0000 7f7995400640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900cf530 0x7f7974006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.081+0000 7f7995400640 1 -- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7974005bc0 msgr2=0x7f7974005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:05.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.081+0000 7f7995400640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7974005bc0 0x7f7974005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.081+0000 7f7995400640 1 -- 172.21.15.49:0/914898459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7974005730 con 0x7f7974006f50 2024-04-15T15:21:05.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.081+0000 7f7994a00640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7974005bc0 0x7f7974005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:05.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.082+0000 7f7995400640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7974006f50 0x7f79740073d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f798800c3e0 tx=0x7f7974007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c0969aa0c046a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:05.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.083+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798800c0e0 con 0x7f7974006f50 2024-04-15T15:21:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.083+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7988030070 con 0x7f7974006f50 2024-04-15T15:21:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.083+0000 7f798f400640 1 -- 172.21.15.49:0/914898459 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880058d0 con 0x7f7974006f50 2024-04-15T15:21:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.083+0000 7f7995e00640 1 -- 172.21.15.49:0/914898459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7974006f50 msgr2=0x7f79740073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.083+0000 7f7995e00640 1 --2- 172.21.15.49:0/914898459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7974006f50 0x7f79740073d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f798800c3e0 tx=0x7f7974007a40 comp rx=0 tx=0).stop 2024-04-15T15:21:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.083+0000 7f7995400640 1 -- 172.21.15.49:0/914898459 reap_dead start 2024-04-15T15:21:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.083+0000 7f7995e00640 1 -- 172.21.15.49:0/914898459 shutdown_connections 2024-04-15T15:21:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.083+0000 7f7995e00640 1 -- 172.21.15.49:0/914898459 >> 172.21.15.49:0/914898459 conn(0x7f79900b23a0 msgr2=0x7f79900b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.083+0000 7f7995e00640 1 -- 172.21.15.49:0/914898459 shutdown_connections 2024-04-15T15:21:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.083+0000 7f7995e00640 1 -- 172.21.15.49:0/914898459 wait complete. 2024-04-15T15:21:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.085+0000 7f7995e00640 1 Processor -- start 2024-04-15T15:21:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.085+0000 7f7995e00640 1 -- start start 2024-04-15T15:21:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.086+0000 7f7995e00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7974006f50 0x7f799014f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.086+0000 7f7995e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79900b4b20 0x7f7990153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.086+0000 7f7995e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79900b4fa0 0x7f79901543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.086+0000 7f7995e00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f79900cd030 con 0x7f79900b4b20 2024-04-15T15:21:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.086+0000 7f7995e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79900cdee0 con 0x7f79900b4fa0 2024-04-15T15:21:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.086+0000 7f7995e00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f79900b03a0 con 0x7f7974006f50 2024-04-15T15:21:05.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.087+0000 7f798fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79900b4b20 0x7f7990153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.087+0000 7f798fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79900b4b20 0x7f7990153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58680/0 (socket says 172.21.15.49:58680) 2024-04-15T15:21:05.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.087+0000 7f798fe00640 1 -- 172.21.15.49:0/2829261052 learned_addr learned my addr 172.21.15.49:0/2829261052 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:05.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.087+0000 7f7994a00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7974006f50 0x7f799014f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.087+0000 7f798fe00640 1 -- 172.21.15.49:0/2829261052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7974006f50 msgr2=0x7f799014f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.087+0000 7f798fe00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7974006f50 0x7f799014f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.087+0000 7f798fe00640 1 -- 172.21.15.49:0/2829261052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79900b4fa0 msgr2=0x7f79901543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.087+0000 7f798fe00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79900b4fa0 0x7f79901543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.087+0000 7f798fe00640 1 -- 172.21.15.49:0/2829261052 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79900b5530 con 0x7f79900b4b20 2024-04-15T15:21:05.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.087+0000 7f798fe00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79900b4b20 0x7f7990153e70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7978011600 tx=0x7f7978011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:05.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.088+0000 7f798cc00640 1 -- 172.21.15.49:0/2829261052 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978015870 con 0x7f79900b4b20 2024-04-15T15:21:05.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.088+0000 7f798cc00640 1 -- 172.21.15.49:0/2829261052 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7978015a10 con 0x7f79900b4b20 2024-04-15T15:21:05.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.088+0000 7f7995e00640 1 -- 172.21.15.49:0/2829261052 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79901f5e20 con 0x7f79900b4b20 2024-04-15T15:21:05.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.088+0000 7f798cc00640 1 -- 172.21.15.49:0/2829261052 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978015dc0 con 0x7f79900b4b20 2024-04-15T15:21:05.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.088+0000 7f7995e00640 1 -- 172.21.15.49:0/2829261052 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79901f5340 con 0x7f79900b4b20 2024-04-15T15:21:05.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.089+0000 7f7995e00640 1 -- 172.21.15.49:0/2829261052 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79900755c0 con 0x7f79900b4b20 2024-04-15T15:21:05.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.090+0000 7f798cc00640 1 -- 172.21.15.49:0/2829261052 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7978013430 con 0x7f79900b4b20 2024-04-15T15:21:05.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.091+0000 7f798cc00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f796c06c9b0 0x7f796c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.092+0000 7f798cc00640 1 -- 172.21.15.49:0/2829261052 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f797808c1a0 con 0x7f79900b4b20 2024-04-15T15:21:05.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.092+0000 7f7994a00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f796c06c9b0 0x7f796c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.093+0000 7f7994a00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f796c06c9b0 0x7f796c06ee70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f799015a430 tx=0x7f798000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:05.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.096+0000 7f798cc00640 1 -- 172.21.15.49:0/2829261052 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7978019030 con 0x7f79900b4b20 2024-04-15T15:21:05.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.301+0000 7f7995e00640 1 -- 172.21.15.49:0/2829261052 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f799015ceb0 con 0x7f796c06c9b0 2024-04-15T15:21:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.302+0000 7f798cc00640 1 -- 172.21.15.49:0/2829261052 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f799015ceb0 con 0x7f796c06c9b0 2024-04-15T15:21:05.305 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 5 to deep-scrub 2024-04-15T15:21:05.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.304+0000 7f7995e00640 1 -- 172.21.15.49:0/2829261052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f796c06c9b0 msgr2=0x7f796c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.304+0000 7f7995e00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f796c06c9b0 0x7f796c06ee70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f799015a430 tx=0x7f798000b040 comp rx=0 tx=0).stop 2024-04-15T15:21:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.304+0000 7f7995e00640 1 -- 172.21.15.49:0/2829261052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79900b4b20 msgr2=0x7f7990153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.304+0000 7f7995e00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79900b4b20 0x7f7990153e70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7978011600 tx=0x7f7978011ac0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.304+0000 7f7995e00640 1 -- 172.21.15.49:0/2829261052 shutdown_connections 2024-04-15T15:21:05.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.304+0000 7f7995e00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f796c06c9b0 0x7f796c06ee70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.304+0000 7f7995e00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79900b4fa0 0x7f79901543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.304+0000 7f7995e00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79900b4b20 0x7f7990153e70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.304+0000 7f7995e00640 1 --2- 172.21.15.49:0/2829261052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7974006f50 0x7f799014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.304+0000 7f7995e00640 1 -- 172.21.15.49:0/2829261052 >> 172.21.15.49:0/2829261052 conn(0x7f79900b23a0 msgr2=0x7f79900af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.305+0000 7f7995e00640 1 -- 172.21.15.49:0/2829261052 shutdown_connections 2024-04-15T15:21:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.305+0000 7f7995e00640 1 -- 172.21.15.49:0/2829261052 wait complete. 2024-04-15T15:21:05.327 INFO:tasks.ceph:Scrubbing osd.6 2024-04-15T15:21:05.327 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.559+0000 7efecd400640 1 Processor -- start 2024-04-15T15:21:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.560+0000 7efecd400640 1 -- start start 2024-04-15T15:21:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.561+0000 7efecd400640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efec80cf510 0x7efec80b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.561+0000 7efecd400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 0x7efec80b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.561+0000 7efecd400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec80d3350 con 0x7efec80d2b90 2024-04-15T15:21:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.561+0000 7efecd400640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7efec80d3f60 con 0x7efec80cf510 2024-04-15T15:21:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.561+0000 7efecd400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7efec80d3630 con 0x7efec80b78e0 2024-04-15T15:21:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.562+0000 7efec7400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7efec80d2b90 0x7efec80cf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42608/0 (socket says 172.21.15.49:42608) 2024-04-15T15:21:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.562+0000 7efec7400640 1 -- 172.21.15.49:0/872867477 learned_addr learned my addr 172.21.15.49:0/872867477 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:21:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.562+0000 7efecca00640 1 --2- 172.21.15.49:0/872867477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 0x7efec80b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.562+0000 7efec7e00640 1 --2- 172.21.15.49:0/872867477 >> v2:172.21.15.107:3300/0 conn(0x7efec80cf510 0x7efec80b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.562+0000 7efec6a00640 1 -- 172.21.15.49:0/872867477 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990431920 0 0) 0x7efec80d3350 con 0x7efec80d2b90 2024-04-15T15:21:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.562+0000 7efec6a00640 1 -- 172.21.15.49:0/872867477 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeac003520 con 0x7efec80d2b90 2024-04-15T15:21:05.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.563+0000 7efecca00640 1 -- 172.21.15.49:0/872867477 >> v2:172.21.15.107:3300/0 conn(0x7efec80cf510 msgr2=0x7efec80b6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:05.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.563+0000 7efecca00640 1 --2- 172.21.15.49:0/872867477 >> v2:172.21.15.107:3300/0 conn(0x7efec80cf510 0x7efec80b6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.563+0000 7efecca00640 1 -- 172.21.15.49:0/872867477 >> v1:172.21.15.49:6789/0 conn(0x7efec80d2b90 legacy=0x7efec80cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.563+0000 7efec6a00640 1 -- 172.21.15.49:0/872867477 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3037978064 0 0) 0x7efeac003520 con 0x7efec80d2b90 2024-04-15T15:21:05.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.563+0000 7efecca00640 1 -- 172.21.15.49:0/872867477 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec80d3910 con 0x7efec80b78e0 2024-04-15T15:21:05.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.564+0000 7efecca00640 1 --2- 172.21.15.49:0/872867477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 0x7efec80b7d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efec0009d90 tx=0x7efec002bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf0d942facc4370a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:05.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.564+0000 7efec6a00640 1 -- 172.21.15.49:0/872867477 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0039070 con 0x7efec80b78e0 2024-04-15T15:21:05.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.565+0000 7efec6a00640 1 -- 172.21.15.49:0/872867477 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec002c8a0 con 0x7efec80b78e0 2024-04-15T15:21:05.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.565+0000 7efec6a00640 1 -- 172.21.15.49:0/872867477 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4001670 con 0x7efec80b78e0 2024-04-15T15:21:05.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.565+0000 7efecd400640 1 -- 172.21.15.49:0/872867477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 msgr2=0x7efec80b7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.565+0000 7efecd400640 1 --2- 172.21.15.49:0/872867477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 0x7efec80b7d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efec0009d90 tx=0x7efec002bd70 comp rx=0 tx=0).stop 2024-04-15T15:21:05.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.565+0000 7efecd400640 1 -- 172.21.15.49:0/872867477 shutdown_connections 2024-04-15T15:21:05.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.565+0000 7efecd400640 1 --2- 172.21.15.49:0/872867477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 0x7efec80b7d70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.565+0000 7efecd400640 1 --2- 172.21.15.49:0/872867477 >> v2:172.21.15.107:3300/0 conn(0x7efec80cf510 0x7efec80b6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.565+0000 7efecd400640 1 -- 172.21.15.49:0/872867477 >> 172.21.15.49:0/872867477 conn(0x7efec80d13a0 msgr2=0x7efec80d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:05.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.565+0000 7efecd400640 1 -- 172.21.15.49:0/872867477 shutdown_connections 2024-04-15T15:21:05.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.565+0000 7efecd400640 1 -- 172.21.15.49:0/872867477 wait complete. 2024-04-15T15:21:05.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.567+0000 7efecd400640 1 Processor -- start 2024-04-15T15:21:05.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.567+0000 7efecd400640 1 -- start start 2024-04-15T15:21:05.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.568+0000 7efecd400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 0x7efec814d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.568+0000 7efecd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec80cf510 0x7efec81e6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.568+0000 7efecd400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efec80d2b90 0x7efec81e6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.568+0000 7efecd400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efec80d3130 con 0x7efec80d2b90 2024-04-15T15:21:05.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.568+0000 7efecd400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efec80d2fb0 con 0x7efec80cf510 2024-04-15T15:21:05.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.568+0000 7efecd400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7efec80d43f0 con 0x7efec80b78e0 2024-04-15T15:21:05.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efecca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efec80d2b90 0x7efec81e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efecca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efec80d2b90 0x7efec81e6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58686/0 (socket says 172.21.15.49:58686) 2024-04-15T15:21:05.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efecca00640 1 -- 172.21.15.49:0/1317859701 learned_addr learned my addr 172.21.15.49:0/1317859701 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:05.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efec7e00640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 0x7efec814d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efec7400640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec80cf510 0x7efec81e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efecca00640 1 -- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 msgr2=0x7efec814d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efecca00640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 0x7efec814d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efecca00640 1 -- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec80cf510 msgr2=0x7efec81e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efecca00640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec80cf510 0x7efec81e6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efecca00640 1 -- 172.21.15.49:0/1317859701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec80b5850 con 0x7efec80d2b90 2024-04-15T15:21:05.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.569+0000 7efec7e00640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec80b78e0 0x7efec814d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:21:05.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.570+0000 7efecca00640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efec80d2b90 0x7efec81e6770 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efec0009d90 tx=0x7efec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:05.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.570+0000 7efebfe00640 1 -- 172.21.15.49:0/1317859701 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0039050 con 0x7efec80d2b90 2024-04-15T15:21:05.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.570+0000 7efebfe00640 1 -- 172.21.15.49:0/1317859701 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec0045070 con 0x7efec80d2b90 2024-04-15T15:21:05.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.570+0000 7efebfe00640 1 -- 172.21.15.49:0/1317859701 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0009450 con 0x7efec80d2b90 2024-04-15T15:21:05.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.570+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec80b5b30 con 0x7efec80d2b90 2024-04-15T15:21:05.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.570+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec80b60f0 con 0x7efec80d2b90 2024-04-15T15:21:05.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.571+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efec80c86c0 con 0x7efec80d2b90 2024-04-15T15:21:05.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.572+0000 7efebfe00640 1 -- 172.21.15.49:0/1317859701 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7efec0040690 con 0x7efec80d2b90 2024-04-15T15:21:05.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.573+0000 7efebfe00640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efea806c980 0x7efea806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.573+0000 7efebfe00640 1 -- 172.21.15.49:0/1317859701 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7efec00ae240 con 0x7efec80d2b90 2024-04-15T15:21:05.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.574+0000 7efebfe00640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] conn(0x7efea80728e0 0x7efea8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.574+0000 7efebfe00640 1 -- 172.21.15.49:0/1317859701 --> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efec0034e50 con 0x7efea80728e0 2024-04-15T15:21:05.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.574+0000 7efebfe00640 1 -- 172.21.15.49:0/1317859701 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efec00b5050 con 0x7efec80d2b90 2024-04-15T15:21:05.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.574+0000 7efec7e00640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efea806c980 0x7efea806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.574+0000 7efec7400640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] conn(0x7efea80728e0 0x7efea8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.574+0000 7efec7e00640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efea806c980 0x7efea806ee40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009af0 tx=0x7efeb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:05.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.575+0000 7efec7400640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] conn(0x7efea80728e0 0x7efea8074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:05.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.578+0000 7efebfe00640 1 -- 172.21.15.49:0/1317859701 <== osd.6 v2:172.21.15.107:6809/4081027681 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efec0034e50 con 0x7efea80728e0 2024-04-15T15:21:05.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.592+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 --> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efec8068b40 con 0x7efea80728e0 2024-04-15T15:21:05.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.593+0000 7efebfe00640 1 -- 172.21.15.49:0/1317859701 <== osd.6 v2:172.21.15.107:6809/4081027681 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efec8068b40 con 0x7efea80728e0 2024-04-15T15:21:05.596 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:21:05.597 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:05.597 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:21:05.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.594+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] conn(0x7efea80728e0 msgr2=0x7efea8074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.594+0000 7efecd400640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.107:6809/4081027681,v1:172.21.15.107:6811/4081027681] conn(0x7efea80728e0 0x7efea8074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.594+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efea806c980 msgr2=0x7efea806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.594+0000 7efecd400640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7efea806c980 0x7efea806ee40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009af0 tx=0x7efeb8008040 comp rx=0 tx=0).stop 2024-04-15T15:21:05.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.594+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efec80d2b90 msgr2=0x7efec81e6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.594+0000 7efecd400640 1 --2- 172.21.15.49:0/1317859701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efec80d2b90 0x7efec81e6770 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efec0009d90 tx=0x7efec0004a40 comp rx=0 tx=0).stop 2024-04-15T15:21:05.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.594+0000 7efecca00640 1 -- 172.21.15.49:0/1317859701 reap_dead start 2024-04-15T15:21:05.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.594+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 shutdown_connections 2024-04-15T15:21:05.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.594+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 >> 172.21.15.49:0/1317859701 conn(0x7efec80d13a0 msgr2=0x7efec80c8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:05.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.595+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 shutdown_connections 2024-04-15T15:21:05.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.595+0000 7efecd400640 1 -- 172.21.15.49:0/1317859701 wait complete. 2024-04-15T15:21:05.615 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-15T15:21:05.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.840+0000 7f8f0d400640 1 Processor -- start 2024-04-15T15:21:05.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.840+0000 7f8f0d400640 1 -- start start 2024-04-15T15:21:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.842+0000 7f8f0d400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f08148050 0x7f8f080b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.842+0000 7f8f0d400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f08162b30 0x7f8f080b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.842+0000 7f8f0d400640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f080afba0 con 0x7f8f080ae790 2024-04-15T15:21:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.842+0000 7f8f0d400640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f8f080aed00 con 0x7f8f08162b30 2024-04-15T15:21:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.842+0000 7f8f0d400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f8f080aefe0 con 0x7f8f08148050 2024-04-15T15:21:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.843+0000 7f8eff400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8f080ae790 0x7f8f08162420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42618/0 (socket says 172.21.15.49:42618) 2024-04-15T15:21:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.843+0000 7f8eff400640 1 -- 172.21.15.49:0/1254702781 learned_addr learned my addr 172.21.15.49:0/1254702781 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:21:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.843+0000 7f8f0ca00640 1 --2- 172.21.15.49:0/1254702781 >> v2:172.21.15.49:3300/0 conn(0x7f8f08162b30 0x7f8f080b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.843+0000 7f8f07e00640 1 --2- 172.21.15.49:0/1254702781 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f08148050 0x7f8f080b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.843+0000 7f8f07400640 1 -- 172.21.15.49:0/1254702781 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4199307470 0 0) 0x7f8f080afba0 con 0x7f8f080ae790 2024-04-15T15:21:05.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.843+0000 7f8f07400640 1 -- 172.21.15.49:0/1254702781 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8eec003520 con 0x7f8f080ae790 2024-04-15T15:21:05.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.843+0000 7f8f0ca00640 1 -- 172.21.15.49:0/1254702781 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f08148050 msgr2=0x7f8f080b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:05.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.843+0000 7f8f0ca00640 1 --2- 172.21.15.49:0/1254702781 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f08148050 0x7f8f080b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.843+0000 7f8f0ca00640 1 -- 172.21.15.49:0/1254702781 >> v1:172.21.15.49:6789/0 conn(0x7f8f080ae790 legacy=0x7f8f08162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.843+0000 7f8f0ca00640 1 -- 172.21.15.49:0/1254702781 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f080af2c0 con 0x7f8f08162b30 2024-04-15T15:21:05.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.845+0000 7f8f0ca00640 1 --2- 172.21.15.49:0/1254702781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f08162b30 0x7f8f080b0360 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009dc0 tx=0x7f8f0002be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6040daf3048acb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:05.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.845+0000 7f8f07400640 1 -- 172.21.15.49:0/1254702781 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00039070 con 0x7f8f08162b30 2024-04-15T15:21:05.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.845+0000 7f8f07400640 1 -- 172.21.15.49:0/1254702781 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0002cc30 con 0x7f8f08162b30 2024-04-15T15:21:05.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.845+0000 7f8f07400640 1 -- 172.21.15.49:0/1254702781 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00034440 con 0x7f8f08162b30 2024-04-15T15:21:05.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.846+0000 7f8f0d400640 1 -- 172.21.15.49:0/1254702781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f08162b30 msgr2=0x7f8f080b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.846+0000 7f8f0d400640 1 --2- 172.21.15.49:0/1254702781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f08162b30 0x7f8f080b0360 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009dc0 tx=0x7f8f0002be80 comp rx=0 tx=0).stop 2024-04-15T15:21:05.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.846+0000 7f8f0d400640 1 -- 172.21.15.49:0/1254702781 shutdown_connections 2024-04-15T15:21:05.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.846+0000 7f8f0d400640 1 --2- 172.21.15.49:0/1254702781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f08162b30 0x7f8f080b0360 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.846+0000 7f8f0d400640 1 --2- 172.21.15.49:0/1254702781 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f08148050 0x7f8f080b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.846+0000 7f8f0d400640 1 -- 172.21.15.49:0/1254702781 >> 172.21.15.49:0/1254702781 conn(0x7f8f080b23a0 msgr2=0x7f8f080b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:05.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.846+0000 7f8f0d400640 1 -- 172.21.15.49:0/1254702781 shutdown_connections 2024-04-15T15:21:05.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.846+0000 7f8f0d400640 1 -- 172.21.15.49:0/1254702781 wait complete. 2024-04-15T15:21:05.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.848+0000 7f8f0d400640 1 Processor -- start 2024-04-15T15:21:05.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.848+0000 7f8f0d400640 1 -- start start 2024-04-15T15:21:05.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.849+0000 7f8f0d400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f080ae790 0x7f8f080ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.849+0000 7f8f0d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f08148050 0x7f8f081469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.849+0000 7f8f0d400640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f08162b30 0x7f8f08146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.849+0000 7f8f0d400640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081485b0 con 0x7f8f080ae790 2024-04-15T15:21:05.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.849+0000 7f8f0d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08148430 con 0x7f8f08148050 2024-04-15T15:21:05.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.849+0000 7f8f0d400640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f8f080b3cc0 con 0x7f8f08162b30 2024-04-15T15:21:05.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.849+0000 7f8f07e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f080ae790 0x7f8f080ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.849+0000 7f8f07e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f080ae790 0x7f8f080ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58702/0 (socket says 172.21.15.49:58702) 2024-04-15T15:21:05.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.849+0000 7f8f07e00640 1 -- 172.21.15.49:0/2880208276 learned_addr learned my addr 172.21.15.49:0/2880208276 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f07e00640 1 -- 172.21.15.49:0/2880208276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f08162b30 msgr2=0x7f8f08146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f0ca00640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f08162b30 0x7f8f08146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8eff400640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f08148050 0x7f8f081469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f07e00640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f08162b30 0x7f8f08146f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f07e00640 1 -- 172.21.15.49:0/2880208276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f08148050 msgr2=0x7f8f081469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f07e00640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f08148050 0x7f8f081469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f07e00640 1 -- 172.21.15.49:0/2880208276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0815f0c0 con 0x7f8f080ae790 2024-04-15T15:21:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8eff400640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f08148050 0x7f8f081469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f07e00640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f080ae790 0x7f8f080ce7d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d830 tx=0x7f8ef800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f04c00640 1 -- 172.21.15.49:0/2880208276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800b6c0 con 0x7f8f080ae790 2024-04-15T15:21:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f04c00640 1 -- 172.21.15.49:0/2880208276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef8019070 con 0x7f8f080ae790 2024-04-15T15:21:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f0d400640 1 -- 172.21.15.49:0/2880208276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0815f3a0 con 0x7f8f080ae790 2024-04-15T15:21:05.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.850+0000 7f8f04c00640 1 -- 172.21.15.49:0/2880208276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800ba80 con 0x7f8f080ae790 2024-04-15T15:21:05.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.851+0000 7f8f0d400640 1 -- 172.21.15.49:0/2880208276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f080ced10 con 0x7f8f080ae790 2024-04-15T15:21:05.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.852+0000 7f8f0d400640 1 -- 172.21.15.49:0/2880208276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f080755c0 con 0x7f8f080ae790 2024-04-15T15:21:05.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.853+0000 7f8f04c00640 1 -- 172.21.15.49:0/2880208276 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8ef801e460 con 0x7f8f080ae790 2024-04-15T15:21:05.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.854+0000 7f8f04c00640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ee406c980 0x7f8ee406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:05.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.854+0000 7f8f04c00640 1 -- 172.21.15.49:0/2880208276 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f8ef808c0b0 con 0x7f8f080ae790 2024-04-15T15:21:05.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.854+0000 7f8eff400640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ee406c980 0x7f8ee406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:05.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.855+0000 7f8eff400640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ee406c980 0x7f8ee406ee40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8f080b0f40 tx=0x7f8ef0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:05.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:05.859+0000 7f8f04c00640 1 -- 172.21.15.49:0/2880208276 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ef805a690 con 0x7f8f080ae790 2024-04-15T15:21:06.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.062+0000 7f8f0d400640 1 -- 172.21.15.49:0/2880208276 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f8f080b3240 con 0x7f8ee406c980 2024-04-15T15:21:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.063+0000 7f8f04c00640 1 -- 172.21.15.49:0/2880208276 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8f080b3240 con 0x7f8ee406c980 2024-04-15T15:21:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 6 to deep-scrub 2024-04-15T15:21:06.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 -- 172.21.15.49:0/2880208276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ee406c980 msgr2=0x7f8ee406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ee406c980 0x7f8ee406ee40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8f080b0f40 tx=0x7f8ef0009040 comp rx=0 tx=0).stop 2024-04-15T15:21:06.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 -- 172.21.15.49:0/2880208276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f080ae790 msgr2=0x7f8f080ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f080ae790 0x7f8f080ce7d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d830 tx=0x7f8ef800dcf0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 -- 172.21.15.49:0/2880208276 shutdown_connections 2024-04-15T15:21:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f8ee406c980 0x7f8ee406ee40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f08162b30 0x7f8f08146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f08148050 0x7f8f081469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 --2- 172.21.15.49:0/2880208276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f080ae790 0x7f8f080ce7d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 -- 172.21.15.49:0/2880208276 >> 172.21.15.49:0/2880208276 conn(0x7f8f080b23a0 msgr2=0x7f8f081442e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 -- 172.21.15.49:0/2880208276 shutdown_connections 2024-04-15T15:21:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.066+0000 7f8f0d400640 1 -- 172.21.15.49:0/2880208276 wait complete. 2024-04-15T15:21:06.087 INFO:tasks.ceph:Scrubbing osd.7 2024-04-15T15:21:06.088 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:21:06.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.325+0000 7fd8b6000640 1 Processor -- start 2024-04-15T15:21:06.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.325+0000 7fd8b6000640 1 -- start start 2024-04-15T15:21:06.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.326+0000 7fd8b6000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd8b00cf510 0x7fd8b00b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.327+0000 7fd8b6000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b00d3350 con 0x7fd8b00b78e0 2024-04-15T15:21:06.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.327+0000 7fd8b6000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b00d3630 con 0x7fd8b00d2b90 2024-04-15T15:21:06.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.327+0000 7fd8b6000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b00d3f60 con 0x7fd8b00cf510 2024-04-15T15:21:06.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.327+0000 7fd8b4c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd8b00cf510 0x7fd8b00b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.327+0000 7fd8b4c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd8b00cf510 0x7fd8b00b6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58712/0 (socket says 172.21.15.49:58712) 2024-04-15T15:21:06.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.327+0000 7fd8b4c00640 1 -- 172.21.15.49:0/1143292338 learned_addr learned my addr 172.21.15.49:0/1143292338 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:06.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.327+0000 7fd8b4c00640 1 -- 172.21.15.49:0/1143292338 >> v1:172.21.15.107:6789/0 conn(0x7fd8b00d2b90 legacy=0x7fd8b00cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:06.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.327+0000 7fd8b4c00640 1 -- 172.21.15.49:0/1143292338 >> v1:172.21.15.49:6789/0 conn(0x7fd8b00b78e0 legacy=0x7fd8b00b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.327+0000 7fd8b4c00640 1 -- 172.21.15.49:0/1143292338 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b00d3910 con 0x7fd8b00cf510 2024-04-15T15:21:06.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.329+0000 7fd8b4c00640 1 --2- 172.21.15.49:0/1143292338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8b00cf510 0x7fd8b00b6b80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009870 tx=0x7fd8a002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b9f62530b81858 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:06.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.329+0000 7fd8af400640 1 -- 172.21.15.49:0/1143292338 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0034070 con 0x7fd8b00cf510 2024-04-15T15:21:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.329+0000 7fd8af400640 1 -- 172.21.15.49:0/1143292338 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a0030860 con 0x7fd8b00cf510 2024-04-15T15:21:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.329+0000 7fd8af400640 1 -- 172.21.15.49:0/1143292338 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0030c30 con 0x7fd8b00cf510 2024-04-15T15:21:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.330+0000 7fd8b6000640 1 -- 172.21.15.49:0/1143292338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8b00cf510 msgr2=0x7fd8b00b6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.330+0000 7fd8b6000640 1 --2- 172.21.15.49:0/1143292338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8b00cf510 0x7fd8b00b6b80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009870 tx=0x7fd8a002b9a0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.330+0000 7fd8b6000640 1 -- 172.21.15.49:0/1143292338 shutdown_connections 2024-04-15T15:21:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.330+0000 7fd8b6000640 1 --2- 172.21.15.49:0/1143292338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8b00cf510 0x7fd8b00b6b80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.330+0000 7fd8b6000640 1 -- 172.21.15.49:0/1143292338 >> 172.21.15.49:0/1143292338 conn(0x7fd8b00d13a0 msgr2=0x7fd8b00d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:06.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.330+0000 7fd8b6000640 1 -- 172.21.15.49:0/1143292338 shutdown_connections 2024-04-15T15:21:06.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.330+0000 7fd8b6000640 1 -- 172.21.15.49:0/1143292338 wait complete. 2024-04-15T15:21:06.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.332+0000 7fd8b6000640 1 Processor -- start 2024-04-15T15:21:06.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.332+0000 7fd8b6000640 1 -- start start 2024-04-15T15:21:06.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8b6000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8b00b78e0 0x7fd8b015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8b6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b00cf510 0x7fd8b0150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8b6000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8b00d2b90 0x7fd8b01f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8b6000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b00d3130 con 0x7fd8b00d2b90 2024-04-15T15:21:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8b6000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b00d2fb0 con 0x7fd8b00cf510 2024-04-15T15:21:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8b6000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b00d43f0 con 0x7fd8b00b78e0 2024-04-15T15:21:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8b5600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8b00d2b90 0x7fd8b01f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8b5600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8b00d2b90 0x7fd8b01f3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58726/0 (socket says 172.21.15.49:58726) 2024-04-15T15:21:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8b5600640 1 -- 172.21.15.49:0/2423146623 learned_addr learned my addr 172.21.15.49:0/2423146623 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8b4c00640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8b00b78e0 0x7fd8b015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.333+0000 7fd8afe00640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b00cf510 0x7fd8b0150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.334+0000 7fd8b5600640 1 -- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8b00b78e0 msgr2=0x7fd8b015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.334+0000 7fd8b5600640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8b00b78e0 0x7fd8b015ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.334+0000 7fd8b5600640 1 -- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b00cf510 msgr2=0x7fd8b0150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.334+0000 7fd8b5600640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b00cf510 0x7fd8b0150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.334+0000 7fd8b5600640 1 -- 172.21.15.49:0/2423146623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b014aa60 con 0x7fd8b00d2b90 2024-04-15T15:21:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.334+0000 7fd8b5600640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8b00d2b90 0x7fd8b01f3c50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c6e0 tx=0x7fd8a800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.334+0000 7fd8acc00640 1 -- 172.21.15.49:0/2423146623 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8003b70 con 0x7fd8b00d2b90 2024-04-15T15:21:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.334+0000 7fd8acc00640 1 -- 172.21.15.49:0/2423146623 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a8003d30 con 0x7fd8b00d2b90 2024-04-15T15:21:06.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.334+0000 7fd8acc00640 1 -- 172.21.15.49:0/2423146623 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8005310 con 0x7fd8b00d2b90 2024-04-15T15:21:06.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.334+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b014ad30 con 0x7fd8b00d2b90 2024-04-15T15:21:06.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.335+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b00b6ca0 con 0x7fd8b00d2b90 2024-04-15T15:21:06.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.335+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd8b0068b40 con 0x7fd8b00d2b90 2024-04-15T15:21:06.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.337+0000 7fd8acc00640 1 -- 172.21.15.49:0/2423146623 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd8a80054b0 con 0x7fd8b00d2b90 2024-04-15T15:21:06.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.337+0000 7fd8acc00640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd88c06c980 0x7fd88c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.338+0000 7fd8acc00640 1 -- 172.21.15.49:0/2423146623 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fd8a808b900 con 0x7fd8b00d2b90 2024-04-15T15:21:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.338+0000 7fd8b4c00640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd88c06c980 0x7fd88c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.338+0000 7fd8acc00640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] conn(0x7fd88c0728e0 0x7fd88c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.338+0000 7fd8acc00640 1 -- 172.21.15.49:0/2423146623 --> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8a8004ec0 con 0x7fd88c0728e0 2024-04-15T15:21:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.338+0000 7fd8acc00640 1 -- 172.21.15.49:0/2423146623 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd8a80902a0 con 0x7fd8b00d2b90 2024-04-15T15:21:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.339+0000 7fd8afe00640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] conn(0x7fd88c0728e0 0x7fd88c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.339+0000 7fd8b4c00640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd88c06c980 0x7fd88c06ee40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0036000 tx=0x7fd8a003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:06.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.340+0000 7fd8afe00640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] conn(0x7fd88c0728e0 0x7fd88c074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:06.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.342+0000 7fd8acc00640 1 -- 172.21.15.49:0/2423146623 <== osd.7 v2:172.21.15.107:6820/876829831 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8a8004ec0 con 0x7fd88c0728e0 2024-04-15T15:21:06.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.357+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 --> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd8b006da40 con 0x7fd88c0728e0 2024-04-15T15:21:06.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.358+0000 7fd8acc00640 1 -- 172.21.15.49:0/2423146623 <== osd.7 v2:172.21.15.107:6820/876829831 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd8b006da40 con 0x7fd88c0728e0 2024-04-15T15:21:06.361 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-15T15:21:06.361 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:21:06.361 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-15T15:21:06.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.358+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] conn(0x7fd88c0728e0 msgr2=0x7fd88c074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.358+0000 7fd8b6000640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.107:6820/876829831,v1:172.21.15.107:6823/876829831] conn(0x7fd88c0728e0 0x7fd88c074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.359+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd88c06c980 msgr2=0x7fd88c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.359+0000 7fd8b6000640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd88c06c980 0x7fd88c06ee40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0036000 tx=0x7fd8a003b040 comp rx=0 tx=0).stop 2024-04-15T15:21:06.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.359+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8b00d2b90 msgr2=0x7fd8b01f3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.359+0000 7fd8b6000640 1 --2- 172.21.15.49:0/2423146623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8b00d2b90 0x7fd8b01f3c50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c6e0 tx=0x7fd8a800cba0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.359+0000 7fd8b5600640 1 -- 172.21.15.49:0/2423146623 reap_dead start 2024-04-15T15:21:06.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.359+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 shutdown_connections 2024-04-15T15:21:06.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.359+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 >> 172.21.15.49:0/2423146623 conn(0x7fd8b00d13a0 msgr2=0x7fd8b006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:06.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.359+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 shutdown_connections 2024-04-15T15:21:06.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.359+0000 7fd8b6000640 1 -- 172.21.15.49:0/2423146623 wait complete. 2024-04-15T15:21:06.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.601+0000 7f97fd000640 1 Processor -- start 2024-04-15T15:21:06.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.601+0000 7f97fd000640 1 -- start start 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.603+0000 7f97fd000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f97f80ae790 0x7f97f8162420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.603+0000 7f97fd000640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f80b3e90 con 0x7f97f8148050 2024-04-15T15:21:06.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.603+0000 7f97fd000640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f80afba0 con 0x7f97f8162960 2024-04-15T15:21:06.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.603+0000 7f97fd000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f97f80aed00 con 0x7f97f80ae790 2024-04-15T15:21:06.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.603+0000 7f97f6a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f97f80ae790 0x7f97f8162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.603+0000 7f97f7400640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f97f8148050 0x7f97f80b3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42638/0 (socket says 172.21.15.49:42638) 2024-04-15T15:21:06.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.603+0000 7f97f7400640 1 -- 172.21.15.49:0/2916323043 learned_addr learned my addr 172.21.15.49:0/2916323043 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:21:06.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.603+0000 7f97f6a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f97f80ae790 0x7f97f8162420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58728/0 (socket says 172.21.15.49:58728) 2024-04-15T15:21:06.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.604+0000 7f97f6a00640 1 -- 172.21.15.49:0/2916323043 >> v1:172.21.15.107:6789/0 conn(0x7f97f8162960 legacy=0x7f97f80b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:06.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.604+0000 7f97f6a00640 1 -- 172.21.15.49:0/2916323043 >> v1:172.21.15.49:6789/0 conn(0x7f97f8148050 legacy=0x7f97f80b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.604+0000 7f97f6a00640 1 -- 172.21.15.49:0/2916323043 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f80aefe0 con 0x7f97f80ae790 2024-04-15T15:21:06.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.605+0000 7f97f6a00640 1 --2- 172.21.15.49:0/2916323043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f80ae790 0x7f97f8162420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f97e40097a0 tx=0x7f97e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979b63bfd6f87c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:06.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.605+0000 7f97f6000640 1 -- 172.21.15.49:0/2916323043 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4035020 con 0x7f97f80ae790 2024-04-15T15:21:06.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.605+0000 7f97f6000640 1 -- 172.21.15.49:0/2916323043 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e4031830 con 0x7f97f80ae790 2024-04-15T15:21:06.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.605+0000 7f97f6000640 1 -- 172.21.15.49:0/2916323043 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4031c00 con 0x7f97f80ae790 2024-04-15T15:21:06.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.606+0000 7f97fd000640 1 -- 172.21.15.49:0/2916323043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f80ae790 msgr2=0x7f97f8162420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.606+0000 7f97fd000640 1 --2- 172.21.15.49:0/2916323043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f80ae790 0x7f97f8162420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f97e40097a0 tx=0x7f97e402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.606+0000 7f97fd000640 1 -- 172.21.15.49:0/2916323043 shutdown_connections 2024-04-15T15:21:06.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.606+0000 7f97fd000640 1 --2- 172.21.15.49:0/2916323043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f80ae790 0x7f97f8162420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.606+0000 7f97fd000640 1 -- 172.21.15.49:0/2916323043 >> 172.21.15.49:0/2916323043 conn(0x7f97f80b23a0 msgr2=0x7f97f80b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:06.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.606+0000 7f97fd000640 1 -- 172.21.15.49:0/2916323043 shutdown_connections 2024-04-15T15:21:06.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.606+0000 7f97fd000640 1 -- 172.21.15.49:0/2916323043 wait complete. 2024-04-15T15:21:06.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.608+0000 7f97fd000640 1 Processor -- start 2024-04-15T15:21:06.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.608+0000 7f97fd000640 1 -- start start 2024-04-15T15:21:06.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.609+0000 7f97fd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f80ae790 0x7f97f815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.609+0000 7f97fd000640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f8148050 0x7f97f80cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.609+0000 7f97fd000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f8162960 0x7f97f80ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.609+0000 7f97fd000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97f8148430 con 0x7f97f8162960 2024-04-15T15:21:06.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.609+0000 7f97fd000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97f80afe40 con 0x7f97f80ae790 2024-04-15T15:21:06.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.609+0000 7f97fd000640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f97f81485b0 con 0x7f97f8148050 2024-04-15T15:21:06.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.609+0000 7f97f7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f8162960 0x7f97f80ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.609+0000 7f97f7e00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f8162960 0x7f97f80ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58744/0 (socket says 172.21.15.49:58744) 2024-04-15T15:21:06.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.609+0000 7f97f7e00640 1 -- 172.21.15.49:0/422485002 learned_addr learned my addr 172.21.15.49:0/422485002 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:06.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.610+0000 7f97f7400640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f80ae790 0x7f97f815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.610+0000 7f97f7e00640 1 -- 172.21.15.49:0/422485002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f8148050 msgr2=0x7f97f80cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.610+0000 7f97f6a00640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f8148050 0x7f97f80cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.610+0000 7f97f7e00640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f8148050 0x7f97f80cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.610+0000 7f97f7e00640 1 -- 172.21.15.49:0/422485002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f80ae790 msgr2=0x7f97f815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.610+0000 7f97f7e00640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f80ae790 0x7f97f815e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.610+0000 7f97f7e00640 1 -- 172.21.15.49:0/422485002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f813e660 con 0x7f97f8162960 2024-04-15T15:21:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.610+0000 7f97f7400640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f80ae790 0x7f97f815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.610+0000 7f97f7e00640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f8162960 0x7f97f80ce250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f97f000b970 tx=0x7f97f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.611+0000 7f97ef400640 1 -- 172.21.15.49:0/422485002 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0019070 con 0x7f97f8162960 2024-04-15T15:21:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.611+0000 7f97ef400640 1 -- 172.21.15.49:0/422485002 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f000cb40 con 0x7f97f8162960 2024-04-15T15:21:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.611+0000 7f97ef400640 1 -- 172.21.15.49:0/422485002 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0014440 con 0x7f97f8162960 2024-04-15T15:21:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.611+0000 7f97fd000640 1 -- 172.21.15.49:0/422485002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f813e940 con 0x7f97f8162960 2024-04-15T15:21:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.611+0000 7f97fd000640 1 -- 172.21.15.49:0/422485002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f813ef00 con 0x7f97f8162960 2024-04-15T15:21:06.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.612+0000 7f97fd000640 1 -- 172.21.15.49:0/422485002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f80755c0 con 0x7f97f8162960 2024-04-15T15:21:06.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.613+0000 7f97ef400640 1 -- 172.21.15.49:0/422485002 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f97f001c070 con 0x7f97f8162960 2024-04-15T15:21:06.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.614+0000 7f97ef400640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97d806c980 0x7f97d806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:06.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.614+0000 7f97ef400640 1 -- 172.21.15.49:0/422485002 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f97f008b4f0 con 0x7f97f8162960 2024-04-15T15:21:06.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.614+0000 7f97f7400640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97d806c980 0x7f97d806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:06.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.615+0000 7f97f7400640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97d806c980 0x7f97d806ee40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f97e8000f60 tx=0x7f97e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:06.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.618+0000 7f97ef400640 1 -- 172.21.15.49:0/422485002 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97f0059c60 con 0x7f97f8162960 2024-04-15T15:21:06.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.827+0000 7f97fd000640 1 -- 172.21.15.49:0/422485002 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f97f80b05e0 con 0x7f97d806c980 2024-04-15T15:21:06.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.827+0000 7f97ef400640 1 -- 172.21.15.49:0/422485002 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f97f80b05e0 con 0x7f97d806c980 2024-04-15T15:21:06.830 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 7 to deep-scrub 2024-04-15T15:21:06.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.830+0000 7f97fd000640 1 -- 172.21.15.49:0/422485002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97d806c980 msgr2=0x7f97d806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.830+0000 7f97fd000640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97d806c980 0x7f97d806ee40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f97e8000f60 tx=0x7f97e8009040 comp rx=0 tx=0).stop 2024-04-15T15:21:06.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.830+0000 7f97fd000640 1 -- 172.21.15.49:0/422485002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f8162960 msgr2=0x7f97f80ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.830+0000 7f97fd000640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f8162960 0x7f97f80ce250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f97f000b970 tx=0x7f97f000be30 comp rx=0 tx=0).stop 2024-04-15T15:21:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.830+0000 7f97fd000640 1 -- 172.21.15.49:0/422485002 shutdown_connections 2024-04-15T15:21:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.830+0000 7f97fd000640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f97d806c980 0x7f97d806ee40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.830+0000 7f97fd000640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97f8148050 0x7f97f80cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.830+0000 7f97fd000640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97f8162960 0x7f97f80ce250 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.830+0000 7f97fd000640 1 --2- 172.21.15.49:0/422485002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97f80ae790 0x7f97f815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.830+0000 7f97fd000640 1 -- 172.21.15.49:0/422485002 >> 172.21.15.49:0/422485002 conn(0x7f97f80b23a0 msgr2=0x7f97f806ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.831+0000 7f97fd000640 1 -- 172.21.15.49:0/422485002 shutdown_connections 2024-04-15T15:21:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:06.831+0000 7f97fd000640 1 -- 172.21.15.49:0/422485002 wait complete. 2024-04-15T15:21:06.853 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:21:07.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.081+0000 7f9918a00640 1 Processor -- start 2024-04-15T15:21:07.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.082+0000 7f9918a00640 1 -- start start 2024-04-15T15:21:07.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.083+0000 7f9918a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99140d1c90 0x7f9914162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:07.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.083+0000 7f9918a00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f99140cf110 0x7f99140d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:07.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.083+0000 7f9918a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99140d0090 con 0x7f9914162020 2024-04-15T15:21:07.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.083+0000 7f9918a00640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 -- 0x7f9914163440 con 0x7f99140cf110 2024-04-15T15:21:07.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.083+0000 7f9918a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f99140b7b30 con 0x7f99140d1c90 2024-04-15T15:21:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.084+0000 7f9913e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f99140cf110 0x7f99140d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.084+0000 7f9913e00640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f99140cf110 0x7f99140d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:42750/0 (socket says 172.21.15.49:42750) 2024-04-15T15:21:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.084+0000 7f9913e00640 1 -- 172.21.15.49:0/966534140 learned_addr learned my addr 172.21.15.49:0/966534140 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.084+0000 7f9912a00640 1 --2- 172.21.15.49:0/966534140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99140d1c90 0x7f9914162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.085+0000 7f9913e00640 1 -- 172.21.15.49:0/966534140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99140d1c90 msgr2=0x7f9914162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.085+0000 7f9913e00640 1 --2- 172.21.15.49:0/966534140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99140d1c90 0x7f9914162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.085+0000 7f9913e00640 1 -- 172.21.15.49:0/966534140 >> v1:172.21.15.107:6789/0 conn(0x7f9914162020 legacy=0x7f99140d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.085+0000 7f9913e00640 1 -- 172.21.15.49:0/966534140 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99140cfad0 con 0x7f99140cf110 2024-04-15T15:21:07.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.086+0000 7f9913e00640 1 --2- 172.21.15.49:0/966534140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99140cf110 0x7f99140d3390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f990c009e50 tx=0x7f990c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ae03a9222ca537b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:07.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.087+0000 7f9912000640 1 -- 172.21.15.49:0/966534140 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c02cb00 con 0x7f99140cf110 2024-04-15T15:21:07.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.087+0000 7f9912000640 1 -- 172.21.15.49:0/966534140 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c02cca0 con 0x7f99140cf110 2024-04-15T15:21:07.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.087+0000 7f9912000640 1 -- 172.21.15.49:0/966534140 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c034610 con 0x7f99140cf110 2024-04-15T15:21:07.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.088+0000 7f9918a00640 1 -- 172.21.15.49:0/966534140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99140cf110 msgr2=0x7f99140d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:07.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.088+0000 7f9918a00640 1 --2- 172.21.15.49:0/966534140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99140cf110 0x7f99140d3390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f990c009e50 tx=0x7f990c02bf60 comp rx=0 tx=0).stop 2024-04-15T15:21:07.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.088+0000 7f9918a00640 1 -- 172.21.15.49:0/966534140 shutdown_connections 2024-04-15T15:21:07.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.088+0000 7f9918a00640 1 --2- 172.21.15.49:0/966534140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99140cf110 0x7f99140d3390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:07.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.088+0000 7f9918a00640 1 --2- 172.21.15.49:0/966534140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99140d1c90 0x7f9914162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:07.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.088+0000 7f9918a00640 1 -- 172.21.15.49:0/966534140 >> 172.21.15.49:0/966534140 conn(0x7f991406cf00 msgr2=0x7f991406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:07.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.088+0000 7f9918a00640 1 -- 172.21.15.49:0/966534140 shutdown_connections 2024-04-15T15:21:07.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.088+0000 7f9918a00640 1 -- 172.21.15.49:0/966534140 wait complete. 2024-04-15T15:21:07.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.090+0000 7f9918a00640 1 Processor -- start 2024-04-15T15:21:07.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.090+0000 7f9918a00640 1 -- start start 2024-04-15T15:21:07.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.091+0000 7f9918a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99140cf110 0x7f99140cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:07.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.091+0000 7f9918a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99140d1c90 0x7f9914154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:07.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.091+0000 7f9918a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9914162020 0x7f99140c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:07.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.091+0000 7f9918a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f99140d2b90 con 0x7f99140cf110 2024-04-15T15:21:07.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.091+0000 7f9918a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f99140cf880 con 0x7f99140d1c90 2024-04-15T15:21:07.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.091+0000 7f9918a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f99140d2e70 con 0x7f9914162020 2024-04-15T15:21:07.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.091+0000 7f9913400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99140cf110 0x7f99140cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:07.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.091+0000 7f9913400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99140cf110 0x7f99140cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58754/0 (socket says 172.21.15.49:58754) 2024-04-15T15:21:07.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.092+0000 7f9913400640 1 -- 172.21.15.49:0/4199491914 learned_addr learned my addr 172.21.15.49:0/4199491914 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:07.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.092+0000 7f9912a00640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99140d1c90 0x7f9914154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:07.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.092+0000 7f9913e00640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9914162020 0x7f99140c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:07.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.092+0000 7f9913400640 1 -- 172.21.15.49:0/4199491914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9914162020 msgr2=0x7f99140c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:07.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.092+0000 7f9913400640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9914162020 0x7f99140c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:07.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.092+0000 7f9913400640 1 -- 172.21.15.49:0/4199491914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99140d1c90 msgr2=0x7f9914154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:07.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.092+0000 7f9913400640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99140d1c90 0x7f9914154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:07.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.092+0000 7f9913400640 1 -- 172.21.15.49:0/4199491914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99140c8f00 con 0x7f99140cf110 2024-04-15T15:21:07.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.092+0000 7f9913400640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99140cf110 0x7f99140cd520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f990400c400 tx=0x7f990400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:07.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.093+0000 7f990b400640 1 -- 172.21.15.49:0/4199491914 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040052d0 con 0x7f99140cf110 2024-04-15T15:21:07.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.093+0000 7f990b400640 1 -- 172.21.15.49:0/4199491914 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904005490 con 0x7f99140cf110 2024-04-15T15:21:07.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.093+0000 7f990b400640 1 -- 172.21.15.49:0/4199491914 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904007260 con 0x7f99140cf110 2024-04-15T15:21:07.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.093+0000 7f9918a00640 1 -- 172.21.15.49:0/4199491914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99140c91a0 con 0x7f99140cf110 2024-04-15T15:21:07.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.093+0000 7f9918a00640 1 -- 172.21.15.49:0/4199491914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99140c9760 con 0x7f99140cf110 2024-04-15T15:21:07.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.094+0000 7f9918a00640 1 -- 172.21.15.49:0/4199491914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99140755c0 con 0x7f99140cf110 2024-04-15T15:21:07.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.095+0000 7f990b400640 1 -- 172.21.15.49:0/4199491914 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9904007400 con 0x7f99140cf110 2024-04-15T15:21:07.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.096+0000 7f990b400640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f98f006c980 0x7f98f006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:07.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.096+0000 7f990b400640 1 -- 172.21.15.49:0/4199491914 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f990408aef0 con 0x7f99140cf110 2024-04-15T15:21:07.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.097+0000 7f9912a00640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f98f006c980 0x7f98f006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:07.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.097+0000 7f9912a00640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f98f006c980 0x7f98f006ee40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f99140d1000 tx=0x7f98fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:07.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.101+0000 7f990b400640 1 -- 172.21.15.49:0/4199491914 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99140d0090 con 0x7f99140cf110 2024-04-15T15:21:07.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.312+0000 7f9918a00640 1 -- 172.21.15.49:0/4199491914 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99140c9930 con 0x7f98f006c980 2024-04-15T15:21:07.316 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:21:07.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.313+0000 7f990b400640 1 -- 172.21.15.49:0/4199491914 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13353 (secure 0 0 0) 0x7f99140c9930 con 0x7f98f006c980 2024-04-15T15:21:07.317 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:21:07.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.316+0000 7f9918a00640 1 -- 172.21.15.49:0/4199491914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f98f006c980 msgr2=0x7f98f006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:07.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.316+0000 7f9918a00640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f98f006c980 0x7f98f006ee40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f99140d1000 tx=0x7f98fc008040 comp rx=0 tx=0).stop 2024-04-15T15:21:07.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.316+0000 7f9918a00640 1 -- 172.21.15.49:0/4199491914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99140cf110 msgr2=0x7f99140cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:07.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.316+0000 7f9918a00640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99140cf110 0x7f99140cd520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f990400c400 tx=0x7f990400c8c0 comp rx=0 tx=0).stop 2024-04-15T15:21:07.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.316+0000 7f9918a00640 1 -- 172.21.15.49:0/4199491914 shutdown_connections 2024-04-15T15:21:07.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.316+0000 7f9918a00640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7f98f006c980 0x7f98f006ee40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:07.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.316+0000 7f9918a00640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9914162020 0x7f99140c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:07.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.316+0000 7f9918a00640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99140d1c90 0x7f9914154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:07.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.316+0000 7f9918a00640 1 --2- 172.21.15.49:0/4199491914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99140cf110 0x7f99140cd520 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:07.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.316+0000 7f9918a00640 1 -- 172.21.15.49:0/4199491914 >> 172.21.15.49:0/4199491914 conn(0x7f991406cf00 msgr2=0x7f991415e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:07.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.317+0000 7f9918a00640 1 -- 172.21.15.49:0/4199491914 shutdown_connections 2024-04-15T15:21:07.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:07.317+0000 7f9918a00640 1 -- 172.21.15.49:0/4199491914 wait complete. 2024-04-15T15:21:07.340 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-15T15:21:06.817632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237480,"kb_used_data":4680,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737240,"statfs":{"total":773094113280,"available":772850933760,"internally_reserved":0,"allocated":4792320,"data_stored":2609296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002465"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":204,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-15T15:20:54.143676+0000","last_change":"2024-04-15T15:18:41.162887+0000","last_active":"2024-04-15T15:20:54.143676+0000","last_peered":"2024-04-15T15:20:54.143676+0000","last_clean":"2024-04-15T15:20:54.143676+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:20:54.143676+0000","last_undegraded":"2024-04-15T15:20:54.143676+0000","last_fullsized":"2024-04-15T15:20:54.143676+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:18:39.644354+0000","last_clean_scrub_stamp":"2024-04-15T15:18:39.644354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:32:00.047414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":768938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":768938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:21:07.343 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-15T15:18:39.644354+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=15, tm_hour=15, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=106, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=15, tm_hour=15, tm_min=20, tm_sec=59, tm_wday=0, tm_yday=106, tm_isdst=0) 2024-04-15T15:21:07.343 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-15T15:21:07.949 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:07.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:27.345 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:21:27.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.581+0000 7fd7cac00640 1 Processor -- start 2024-04-15T15:21:27.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.581+0000 7fd7cac00640 1 -- start start 2024-04-15T15:21:27.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.583+0000 7fd7cac00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd7c406cb50 0x7fd7c4073180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.583+0000 7fd7cac00640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c4162760 con 0x7fd7c4159240 2024-04-15T15:21:27.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.583+0000 7fd7cac00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c41629d0 con 0x7fd7c40736c0 2024-04-15T15:21:27.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.583+0000 7fd7cac00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c40cd8f0 con 0x7fd7c406cb50 2024-04-15T15:21:27.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.584+0000 7fd7c3e00640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd7c4159240 0x7fd7c415fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48102/0 (socket says 172.21.15.49:48102) 2024-04-15T15:21:27.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.584+0000 7fd7c3e00640 1 -- 172.21.15.49:0/2748506836 learned_addr learned my addr 172.21.15.49:0/2748506836 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-04-15T15:21:27.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.584+0000 7fd7c3400640 1 --2- 172.21.15.49:0/2748506836 >> v2:172.21.15.49:3300/0 conn(0x7fd7c406cb50 0x7fd7c4073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.584+0000 7fd7c3400640 1 -- 172.21.15.49:0/2748506836 >> v1:172.21.15.107:6789/0 conn(0x7fd7c40736c0 legacy=0x7fd7c40aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:27.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.584+0000 7fd7c3400640 1 -- 172.21.15.49:0/2748506836 >> v1:172.21.15.49:6789/0 conn(0x7fd7c4159240 legacy=0x7fd7c415fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.585+0000 7fd7c3400640 1 -- 172.21.15.49:0/2748506836 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c4162cb0 con 0x7fd7c406cb50 2024-04-15T15:21:27.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.584+0000 7fd7c2a00640 1 -- 172.21.15.49:0/2748506836 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2455279628 0 0) 0x7fd7c4162760 con 0x7fd7c4159240 2024-04-15T15:21:27.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.586+0000 7fd7c3400640 1 --2- 172.21.15.49:0/2748506836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c406cb50 0x7fd7c4073180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00097a0 tx=0x7fd7b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb9f613ccaed2e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:27.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.586+0000 7fd7c2a00640 1 -- 172.21.15.49:0/2748506836 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0035020 con 0x7fd7c406cb50 2024-04-15T15:21:27.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.586+0000 7fd7c2a00640 1 -- 172.21.15.49:0/2748506836 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b0031830 con 0x7fd7c406cb50 2024-04-15T15:21:27.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.586+0000 7fd7c2a00640 1 -- 172.21.15.49:0/2748506836 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0031c00 con 0x7fd7c406cb50 2024-04-15T15:21:27.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.587+0000 7fd7cac00640 1 -- 172.21.15.49:0/2748506836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c406cb50 msgr2=0x7fd7c4073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.587+0000 7fd7cac00640 1 --2- 172.21.15.49:0/2748506836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c406cb50 0x7fd7c4073180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00097a0 tx=0x7fd7b002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.587+0000 7fd7cac00640 1 -- 172.21.15.49:0/2748506836 shutdown_connections 2024-04-15T15:21:27.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.587+0000 7fd7cac00640 1 --2- 172.21.15.49:0/2748506836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c406cb50 0x7fd7c4073180 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.587+0000 7fd7cac00640 1 -- 172.21.15.49:0/2748506836 >> 172.21.15.49:0/2748506836 conn(0x7fd7c406f860 msgr2=0x7fd7c4072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:27.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.587+0000 7fd7cac00640 1 -- 172.21.15.49:0/2748506836 shutdown_connections 2024-04-15T15:21:27.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.587+0000 7fd7cac00640 1 -- 172.21.15.49:0/2748506836 wait complete. 2024-04-15T15:21:27.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.589+0000 7fd7cac00640 1 Processor -- start 2024-04-15T15:21:27.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.589+0000 7fd7cac00640 1 -- start start 2024-04-15T15:21:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.590+0000 7fd7cac00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7c406cb50 0x7fd7c41f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.590+0000 7fd7cac00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c40736c0 0x7fd7c40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.590+0000 7fd7cac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c4159240 0x7fd7c40b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.590+0000 7fd7cac00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c40af5a0 con 0x7fd7c40736c0 2024-04-15T15:21:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.590+0000 7fd7cac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c40af420 con 0x7fd7c4159240 2024-04-15T15:21:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.590+0000 7fd7cac00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c40af720 con 0x7fd7c406cb50 2024-04-15T15:21:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.590+0000 7fd7c3400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c40736c0 0x7fd7c40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.590+0000 7fd7c3400640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c40736c0 0x7fd7c40b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50524/0 (socket says 172.21.15.49:50524) 2024-04-15T15:21:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.591+0000 7fd7c3400640 1 -- 172.21.15.49:0/1777211579 learned_addr learned my addr 172.21.15.49:0/1777211579 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.591+0000 7fd7c8c00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c4159240 0x7fd7c40b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.591+0000 7fd7c3400640 1 -- 172.21.15.49:0/1777211579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7c406cb50 msgr2=0x7fd7c41f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.591+0000 7fd7c3e00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7c406cb50 0x7fd7c41f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.591+0000 7fd7c3400640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7c406cb50 0x7fd7c41f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.591+0000 7fd7c3400640 1 -- 172.21.15.49:0/1777211579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c4159240 msgr2=0x7fd7c40b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.591+0000 7fd7c3400640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c4159240 0x7fd7c40b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.591+0000 7fd7c3400640 1 -- 172.21.15.49:0/1777211579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c4159830 con 0x7fd7c40736c0 2024-04-15T15:21:27.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.591+0000 7fd7c8c00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c4159240 0x7fd7c40b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:21:27.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.591+0000 7fd7c3400640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c40736c0 0x7fd7c40b4f60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00098c0 tx=0x7fd7b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:27.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.592+0000 7fd7bbe00640 1 -- 172.21.15.49:0/1777211579 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0041070 con 0x7fd7c40736c0 2024-04-15T15:21:27.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.592+0000 7fd7bbe00640 1 -- 172.21.15.49:0/1777211579 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b003aa30 con 0x7fd7c40736c0 2024-04-15T15:21:27.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.592+0000 7fd7bbe00640 1 -- 172.21.15.49:0/1777211579 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0035040 con 0x7fd7c40736c0 2024-04-15T15:21:27.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.592+0000 7fd7cac00640 1 -- 172.21.15.49:0/1777211579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c4159b10 con 0x7fd7c40736c0 2024-04-15T15:21:27.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.592+0000 7fd7cac00640 1 -- 172.21.15.49:0/1777211579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c415a0d0 con 0x7fd7c40736c0 2024-04-15T15:21:27.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.593+0000 7fd7cac00640 1 -- 172.21.15.49:0/1777211579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c4068c80 con 0x7fd7c40736c0 2024-04-15T15:21:27.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.594+0000 7fd7bbe00640 1 -- 172.21.15.49:0/1777211579 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd7b004c060 con 0x7fd7c40736c0 2024-04-15T15:21:27.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.595+0000 7fd7bbe00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd7a406c980 0x7fd7a406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:27.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.595+0000 7fd7bbe00640 1 -- 172.21.15.49:0/1777211579 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fd7b00ad490 con 0x7fd7c40736c0 2024-04-15T15:21:27.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.595+0000 7fd7c3e00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd7a406c980 0x7fd7a406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.596+0000 7fd7c3e00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd7a406c980 0x7fd7a406ee40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4000f60 tx=0x7fd7b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:27.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.600+0000 7fd7bbe00640 1 -- 172.21.15.49:0/1777211579 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd7b007ba70 con 0x7fd7c40736c0 2024-04-15T15:21:27.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.804+0000 7fd7cac00640 1 -- 172.21.15.49:0/1777211579 --> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7c40680c0 con 0x7fd7a406c980 2024-04-15T15:21:27.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.805+0000 7fd7bbe00640 1 -- 172.21.15.49:0/1777211579 <== mgr.4101 v2:172.21.15.49:6832/2558964566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7fd7c40680c0 con 0x7fd7a406c980 2024-04-15T15:21:27.807 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:21:27.807 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-15T15:21:27.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 -- 172.21.15.49:0/1777211579 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd7a406c980 msgr2=0x7fd7a406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd7a406c980 0x7fd7a406ee40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4000f60 tx=0x7fd7b4009040 comp rx=0 tx=0).stop 2024-04-15T15:21:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 -- 172.21.15.49:0/1777211579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c40736c0 msgr2=0x7fd7c40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c40736c0 0x7fd7c40b4f60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00098c0 tx=0x7fd7b00046c0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 -- 172.21.15.49:0/1777211579 shutdown_connections 2024-04-15T15:21:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fd7a406c980 0x7fd7a406ee40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c4159240 0x7fd7c40b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7c40736c0 0x7fd7c40b4f60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 --2- 172.21.15.49:0/1777211579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7c406cb50 0x7fd7c41f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 -- 172.21.15.49:0/1777211579 >> 172.21.15.49:0/1777211579 conn(0x7fd7c406f860 msgr2=0x7fd7c4156d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 -- 172.21.15.49:0/1777211579 shutdown_connections 2024-04-15T15:21:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:27.808+0000 7fd7cac00640 1 -- 172.21.15.49:0/1777211579 wait complete. 2024-04-15T15:21:27.828 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-04-15T15:21:26.824484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237472,"kb_used_data":4680,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737248,"statfs":{"total":773094113280,"available":772850941952,"internally_reserved":0,"allocated":4792320,"data_stored":2609296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004476"},"pg_stats":[{"pgid":"1.0","version":"39'119","reported_seq":215,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-15T15:21:06.924475+0000","last_change":"2024-04-15T15:21:06.924544+0000","last_active":"2024-04-15T15:21:06.924475+0000","last_peered":"2024-04-15T15:21:06.924475+0000","last_clean":"2024-04-15T15:21:06.924475+0000","last_became_active":"2024-04-15T15:18:41.160763+0000","last_became_peered":"2024-04-15T15:18:41.160763+0000","last_unstale":"2024-04-15T15:21:06.924475+0000","last_undegraded":"2024-04-15T15:21:06.924475+0000","last_fullsized":"2024-04-15T15:21:06.924475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'119","last_scrub_stamp":"2024-04-15T15:21:06.924280+0000","last_deep_scrub":"39'119","last_deep_scrub_stamp":"2024-04-15T15:21:06.924280+0000","last_clean_scrub_stamp":"2024-04-15T15:21:06.924280+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:39:39.732266+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1044480,"data_stored":768938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":178570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1044480,"data_stored":768938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:21:27.829 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-15T15:21:28.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.048+0000 7fce65600640 1 Processor -- start 2024-04-15T15:21:28.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.048+0000 7fce65600640 1 -- start start 2024-04-15T15:21:28.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.049+0000 7fce65600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce601461c0 0x7fce60071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.050+0000 7fce65600640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fce600cebb0 0x7fce6006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.050+0000 7fce65600640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce600cd2d0 con 0x7fce60072420 2024-04-15T15:21:28.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.050+0000 7fce65600640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fce600cf1d0 con 0x7fce600cebb0 2024-04-15T15:21:28.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.050+0000 7fce65600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fce60072880 con 0x7fce601461c0 2024-04-15T15:21:28.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.050+0000 7fce64c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fce600cebb0 0x7fce6006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.050+0000 7fce64c00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fce600cebb0 0x7fce6006df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50534/0 (socket says 172.21.15.49:50534) 2024-04-15T15:21:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.050+0000 7fce64c00640 1 -- 172.21.15.49:0/2250119146 learned_addr learned my addr 172.21.15.49:0/2250119146 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.050+0000 7fce5fe00640 1 --2- 172.21.15.49:0/2250119146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce601461c0 0x7fce60071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.051+0000 7fce64c00640 1 -- 172.21.15.49:0/2250119146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce601461c0 msgr2=0x7fce60071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.051+0000 7fce64c00640 1 --2- 172.21.15.49:0/2250119146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce601461c0 0x7fce60071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.051+0000 7fce64c00640 1 -- 172.21.15.49:0/2250119146 >> v1:172.21.15.107:6789/0 conn(0x7fce60072420 legacy=0x7fce600ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.051+0000 7fce64c00640 1 -- 172.21.15.49:0/2250119146 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce600cd5b0 con 0x7fce600cebb0 2024-04-15T15:21:28.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.052+0000 7fce64c00640 1 --2- 172.21.15.49:0/2250119146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce600cebb0 0x7fce6006df20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce5802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f559cd81e03eaf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.052+0000 7fce5f400640 1 -- 172.21.15.49:0/2250119146 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5802cb00 con 0x7fce600cebb0 2024-04-15T15:21:28.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.052+0000 7fce5f400640 1 -- 172.21.15.49:0/2250119146 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce5802cca0 con 0x7fce600cebb0 2024-04-15T15:21:28.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.052+0000 7fce5f400640 1 -- 172.21.15.49:0/2250119146 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58034610 con 0x7fce600cebb0 2024-04-15T15:21:28.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.053+0000 7fce65600640 1 -- 172.21.15.49:0/2250119146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce600cebb0 msgr2=0x7fce6006df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.053+0000 7fce65600640 1 --2- 172.21.15.49:0/2250119146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce600cebb0 0x7fce6006df20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce5802bf60 comp rx=0 tx=0).stop 2024-04-15T15:21:28.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.053+0000 7fce65600640 1 -- 172.21.15.49:0/2250119146 shutdown_connections 2024-04-15T15:21:28.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.053+0000 7fce65600640 1 --2- 172.21.15.49:0/2250119146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce600cebb0 0x7fce6006df20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.053+0000 7fce65600640 1 --2- 172.21.15.49:0/2250119146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce601461c0 0x7fce60071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.053+0000 7fce65600640 1 -- 172.21.15.49:0/2250119146 >> 172.21.15.49:0/2250119146 conn(0x7fce6006b930 msgr2=0x7fce6006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.053+0000 7fce65600640 1 -- 172.21.15.49:0/2250119146 shutdown_connections 2024-04-15T15:21:28.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.053+0000 7fce65600640 1 -- 172.21.15.49:0/2250119146 wait complete. 2024-04-15T15:21:28.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.056+0000 7fce65600640 1 Processor -- start 2024-04-15T15:21:28.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.056+0000 7fce65600640 1 -- start start 2024-04-15T15:21:28.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.057+0000 7fce65600640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce60072420 0x7fce60135080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.057+0000 7fce65600640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce600cebb0 0x7fce600b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.057+0000 7fce65600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce601461c0 0x7fce60142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.057+0000 7fce65600640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fce600cdb70 con 0x7fce60072420 2024-04-15T15:21:28.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.057+0000 7fce65600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fce600cd080 con 0x7fce601461c0 2024-04-15T15:21:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.057+0000 7fce65600640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fce6006e460 con 0x7fce600cebb0 2024-04-15T15:21:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.057+0000 7fce5fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce60072420 0x7fce60135080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.057+0000 7fce5fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce60072420 0x7fce60135080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50550/0 (socket says 172.21.15.49:50550) 2024-04-15T15:21:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.057+0000 7fce5fe00640 1 -- 172.21.15.49:0/1529155472 learned_addr learned my addr 172.21.15.49:0/1529155472 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.058+0000 7fce57400640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce600cebb0 0x7fce600b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.058+0000 7fce64c00640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce601461c0 0x7fce60142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.058+0000 7fce5fe00640 1 -- 172.21.15.49:0/1529155472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce600cebb0 msgr2=0x7fce600b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.058+0000 7fce5fe00640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce600cebb0 0x7fce600b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.058+0000 7fce5fe00640 1 -- 172.21.15.49:0/1529155472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce601461c0 msgr2=0x7fce60142860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.058+0000 7fce5fe00640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce601461c0 0x7fce60142860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.058+0000 7fce5fe00640 1 -- 172.21.15.49:0/1529155472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce600b33d0 con 0x7fce60072420 2024-04-15T15:21:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.058+0000 7fce5fe00640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce60072420 0x7fce60135080 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e870 tx=0x7fce5000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.059+0000 7fce5cc00640 1 -- 172.21.15.49:0/1529155472 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50017070 con 0x7fce60072420 2024-04-15T15:21:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.059+0000 7fce5cc00640 1 -- 172.21.15.49:0/1529155472 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce5000c8e0 con 0x7fce60072420 2024-04-15T15:21:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.059+0000 7fce65600640 1 -- 172.21.15.49:0/1529155472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce601355c0 con 0x7fce60072420 2024-04-15T15:21:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.059+0000 7fce5cc00640 1 -- 172.21.15.49:0/1529155472 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5000ccd0 con 0x7fce60072420 2024-04-15T15:21:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.059+0000 7fce65600640 1 -- 172.21.15.49:0/1529155472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce600b3950 con 0x7fce60072420 2024-04-15T15:21:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.060+0000 7fce65600640 1 -- 172.21.15.49:0/1529155472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce600755c0 con 0x7fce60072420 2024-04-15T15:21:28.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.061+0000 7fce5cc00640 1 -- 172.21.15.49:0/1529155472 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fce5001a070 con 0x7fce60072420 2024-04-15T15:21:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.062+0000 7fce5cc00640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fce3c06c9b0 0x7fce3c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.062+0000 7fce5cc00640 1 -- 172.21.15.49:0/1529155472 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fce5008b770 con 0x7fce60072420 2024-04-15T15:21:28.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.063+0000 7fce57400640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fce3c06c9b0 0x7fce3c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.063+0000 7fce57400640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fce3c06c9b0 0x7fce3c06ee70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fce600af520 tx=0x7fce48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.067+0000 7fce5cc00640 1 -- 172.21.15.49:0/1529155472 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce50059de0 con 0x7fce60072420 2024-04-15T15:21:28.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.271+0000 7fce65600640 1 -- 172.21.15.49:0/1529155472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fce60072e40 con 0x7fce60072420 2024-04-15T15:21:28.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.278+0000 7fce5cc00640 1 -- 172.21.15.49:0/1529155472 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7fce500596e0 con 0x7fce60072420 2024-04-15T15:21:28.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.281+0000 7fce5cc00640 1 -- 172.21.15.49:0/1529155472 <== mon.0 v2:172.21.15.49:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fce5005da90 con 0x7fce60072420 2024-04-15T15:21:28.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.283+0000 7fce65600640 1 -- 172.21.15.49:0/1529155472 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fce3c06c9b0 msgr2=0x7fce3c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.283+0000 7fce65600640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fce3c06c9b0 0x7fce3c06ee70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fce600af520 tx=0x7fce48009040 comp rx=0 tx=0).stop 2024-04-15T15:21:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.283+0000 7fce65600640 1 -- 172.21.15.49:0/1529155472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce60072420 msgr2=0x7fce60135080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.283+0000 7fce65600640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce60072420 0x7fce60135080 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e870 tx=0x7fce5000ed30 comp rx=0 tx=0).stop 2024-04-15T15:21:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.284+0000 7fce65600640 1 -- 172.21.15.49:0/1529155472 shutdown_connections 2024-04-15T15:21:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.284+0000 7fce65600640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fce3c06c9b0 0x7fce3c06ee70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.284+0000 7fce65600640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce600cebb0 0x7fce600b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.284+0000 7fce65600640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce601461c0 0x7fce60142860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.284+0000 7fce65600640 1 --2- 172.21.15.49:0/1529155472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce60072420 0x7fce60135080 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.284+0000 7fce65600640 1 -- 172.21.15.49:0/1529155472 >> 172.21.15.49:0/1529155472 conn(0x7fce6006b930 msgr2=0x7fce6015f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.284+0000 7fce65600640 1 -- 172.21.15.49:0/1529155472 shutdown_connections 2024-04-15T15:21:28.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.284+0000 7fce65600640 1 -- 172.21.15.49:0/1529155472 wait complete. 2024-04-15T15:21:28.309 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-15T15:21:28.309 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:21:28.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.530+0000 7fca95a00640 1 Processor -- start 2024-04-15T15:21:28.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.530+0000 7fca95a00640 1 -- start start 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.531+0000 7fca95a00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fca900d1c90 0x7fca90162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.531+0000 7fca95a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca900cf110 0x7fca900d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.531+0000 7fca95a00640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca900d0090 con 0x7fca90162020 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.532+0000 7fca95a00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fca90163440 con 0x7fca900d1c90 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.532+0000 7fca95a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fca900b7b30 con 0x7fca900cf110 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.532+0000 7fca8f400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fca900d1c90 0x7fca90162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.532+0000 7fca8f400640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fca900d1c90 0x7fca90162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50554/0 (socket says 172.21.15.49:50554) 2024-04-15T15:21:28.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.532+0000 7fca8f400640 1 -- 172.21.15.49:0/3094785254 learned_addr learned my addr 172.21.15.49:0/3094785254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:28.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.532+0000 7fca95000640 1 --2- 172.21.15.49:0/3094785254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca900cf110 0x7fca900d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.532+0000 7fca8f400640 1 -- 172.21.15.49:0/3094785254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca900cf110 msgr2=0x7fca900d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:28.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.532+0000 7fca8f400640 1 --2- 172.21.15.49:0/3094785254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca900cf110 0x7fca900d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.532+0000 7fca8f400640 1 -- 172.21.15.49:0/3094785254 >> v1:172.21.15.107:6789/0 conn(0x7fca90162020 legacy=0x7fca900d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:21:28.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.532+0000 7fca8f400640 1 -- 172.21.15.49:0/3094785254 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca900cfad0 con 0x7fca900d1c90 2024-04-15T15:21:28.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.533+0000 7fca8f400640 1 --2- 172.21.15.49:0/3094785254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900d1c90 0x7fca90162f00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009870 tx=0x7fca7c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b297321afe61bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.534+0000 7fca8ea00640 1 -- 172.21.15.49:0/3094785254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c035020 con 0x7fca900d1c90 2024-04-15T15:21:28.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.534+0000 7fca8ea00640 1 -- 172.21.15.49:0/3094785254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fca7c031640 con 0x7fca900d1c90 2024-04-15T15:21:28.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.534+0000 7fca8ea00640 1 -- 172.21.15.49:0/3094785254 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c02d040 con 0x7fca900d1c90 2024-04-15T15:21:28.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.535+0000 7fca95a00640 1 -- 172.21.15.49:0/3094785254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900d1c90 msgr2=0x7fca90162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.535+0000 7fca95a00640 1 --2- 172.21.15.49:0/3094785254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900d1c90 0x7fca90162f00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009870 tx=0x7fca7c02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.535+0000 7fca95a00640 1 -- 172.21.15.49:0/3094785254 shutdown_connections 2024-04-15T15:21:28.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.535+0000 7fca95a00640 1 --2- 172.21.15.49:0/3094785254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca900cf110 0x7fca900d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.535+0000 7fca95a00640 1 --2- 172.21.15.49:0/3094785254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900d1c90 0x7fca90162f00 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.535+0000 7fca95a00640 1 -- 172.21.15.49:0/3094785254 >> 172.21.15.49:0/3094785254 conn(0x7fca9006cf00 msgr2=0x7fca9006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.535+0000 7fca95a00640 1 -- 172.21.15.49:0/3094785254 shutdown_connections 2024-04-15T15:21:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.535+0000 7fca95a00640 1 -- 172.21.15.49:0/3094785254 wait complete. 2024-04-15T15:21:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.538+0000 7fca95a00640 1 Processor -- start 2024-04-15T15:21:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.538+0000 7fca95a00640 1 -- start start 2024-04-15T15:21:28.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.539+0000 7fca95a00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900cf110 0x7fca901f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.539+0000 7fca95a00640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca900d1c90 0x7fca900c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.539+0000 7fca95a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca90162020 0x7fca900cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.539+0000 7fca95a00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fca900d2b90 con 0x7fca900cf110 2024-04-15T15:21:28.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.539+0000 7fca95a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fca900cf880 con 0x7fca90162020 2024-04-15T15:21:28.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.539+0000 7fca95a00640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fca900d2e70 con 0x7fca900d1c90 2024-04-15T15:21:28.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.540+0000 7fca8fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900cf110 0x7fca901f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.540+0000 7fca8fe00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900cf110 0x7fca901f7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50564/0 (socket says 172.21.15.49:50564) 2024-04-15T15:21:28.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.540+0000 7fca8fe00640 1 -- 172.21.15.49:0/3567934993 learned_addr learned my addr 172.21.15.49:0/3567934993 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-15T15:21:28.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.540+0000 7fca95000640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca90162020 0x7fca900cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.540+0000 7fca8f400640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca900d1c90 0x7fca900c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.540+0000 7fca8fe00640 1 -- 172.21.15.49:0/3567934993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca900d1c90 msgr2=0x7fca900c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.540+0000 7fca8fe00640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca900d1c90 0x7fca900c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.540+0000 7fca8fe00640 1 -- 172.21.15.49:0/3567934993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca90162020 msgr2=0x7fca900cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.540+0000 7fca8fe00640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca90162020 0x7fca900cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.540+0000 7fca8fe00640 1 -- 172.21.15.49:0/3567934993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca900c95f0 con 0x7fca900cf110 2024-04-15T15:21:28.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.541+0000 7fca8fe00640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900cf110 0x7fca901f7ac0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fca8000b3f0 tx=0x7fca8000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.541+0000 7fca87e00640 1 -- 172.21.15.49:0/3567934993 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80015020 con 0x7fca900cf110 2024-04-15T15:21:28.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.541+0000 7fca87e00640 1 -- 172.21.15.49:0/3567934993 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fca80011830 con 0x7fca900cf110 2024-04-15T15:21:28.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.541+0000 7fca87e00640 1 -- 172.21.15.49:0/3567934993 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80011c40 con 0x7fca900cf110 2024-04-15T15:21:28.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.541+0000 7fca95a00640 1 -- 172.21.15.49:0/3567934993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca900c98d0 con 0x7fca900cf110 2024-04-15T15:21:28.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.541+0000 7fca95a00640 1 -- 172.21.15.49:0/3567934993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca900c9e90 con 0x7fca900cf110 2024-04-15T15:21:28.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.542+0000 7fca95a00640 1 -- 172.21.15.49:0/3567934993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca900755c0 con 0x7fca900cf110 2024-04-15T15:21:28.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.543+0000 7fca87e00640 1 -- 172.21.15.49:0/3567934993 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fca80018020 con 0x7fca900cf110 2024-04-15T15:21:28.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.544+0000 7fca87e00640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fca7006c980 0x7fca7006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:21:28.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.545+0000 7fca87e00640 1 -- 172.21.15.49:0/3567934993 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fca8008b650 con 0x7fca900cf110 2024-04-15T15:21:28.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.545+0000 7fca8f400640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fca7006c980 0x7fca7006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:21:28.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.546+0000 7fca8f400640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fca7006c980 0x7fca7006ee40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca7c036000 tx=0x7fca7c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:21:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.549+0000 7fca87e00640 1 -- 172.21.15.49:0/3567934993 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca8001f030 con 0x7fca900cf110 2024-04-15T15:21:28.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.795+0000 7fca95a00640 1 -- 172.21.15.49:0/3567934993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca900b7650 con 0x7fca900cf110 2024-04-15T15:21:28.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.796+0000 7fca87e00640 1 -- 172.21.15.49:0/3567934993 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+3875 (secure 0 0 0) 0x7fca800172c0 con 0x7fca900cf110 2024-04-15T15:21:28.798 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-15T15:21:28.798 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1669437860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1669437860},{"type":"v1","addr":"172.21.15.107:6833","nonce":1669437860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3218959504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3218959504},{"type":"v1","addr":"172.21.15.107:6835","nonce":3218959504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5064,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3214547060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3214547060},{"type":"v1","addr":"172.21.15.49:6835","nonce":3214547060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5703,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2806231240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2806231240},{"type":"v1","addr":"172.21.15.49:6837","nonce":2806231240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[]} 2024-04-15T15:21:28.799 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 25 2024-04-15T15:21:28.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 -- 172.21.15.49:0/3567934993 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fca7006c980 msgr2=0x7fca7006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fca7006c980 0x7fca7006ee40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca7c036000 tx=0x7fca7c03c040 comp rx=0 tx=0).stop 2024-04-15T15:21:28.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 -- 172.21.15.49:0/3567934993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900cf110 msgr2=0x7fca901f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:21:28.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900cf110 0x7fca901f7ac0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fca8000b3f0 tx=0x7fca8000b8b0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 -- 172.21.15.49:0/3567934993 shutdown_connections 2024-04-15T15:21:28.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.49:6832/2558964566,v1:172.21.15.49:6833/2558964566] conn(0x7fca7006c980 0x7fca7006ee40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca900d1c90 0x7fca900c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca90162020 0x7fca900cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 --2- 172.21.15.49:0/3567934993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca900cf110 0x7fca901f7ac0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:21:28.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 -- 172.21.15.49:0/3567934993 >> 172.21.15.49:0/3567934993 conn(0x7fca9006cf00 msgr2=0x7fca900d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:21:28.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 -- 172.21.15.49:0/3567934993 shutdown_connections 2024-04-15T15:21:28.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-15T15:21:28.799+0000 7fca95a00640 1 -- 172.21.15.49:0/3567934993 wait complete. 2024-04-15T15:21:28.822 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-15T15:21:28.822 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-15T15:21:28.823 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-15T15:21:28.823 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:28.845 INFO:tasks.ceph.mds.a:Stopped 2024-04-15T15:21:28.845 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-15T15:21:28.845 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:28.909 INFO:tasks.ceph.mds.c:Stopped 2024-04-15T15:21:28.909 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-15T15:21:28.909 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:28.930 INFO:tasks.ceph.mds.b:Stopped 2024-04-15T15:21:28.930 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-15T15:21:28.930 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.002 INFO:tasks.ceph.mds.d:Stopped 2024-04-15T15:21:29.002 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-15T15:21:29.002 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-15T15:21:29.003 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.075 INFO:tasks.ceph.osd.0:Stopped 2024-04-15T15:21:29.075 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-15T15:21:29.076 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.135 INFO:tasks.ceph.osd.1:Stopped 2024-04-15T15:21:29.135 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-15T15:21:29.135 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.202 INFO:tasks.ceph.osd.2:Stopped 2024-04-15T15:21:29.202 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-15T15:21:29.202 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.253 INFO:tasks.ceph.osd.3:Stopped 2024-04-15T15:21:29.253 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-15T15:21:29.254 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.306 INFO:tasks.ceph.osd.4:Stopped 2024-04-15T15:21:29.306 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-15T15:21:29.307 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.375 INFO:tasks.ceph.osd.5:Stopped 2024-04-15T15:21:29.375 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-15T15:21:29.375 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.423 INFO:tasks.ceph.osd.6:Stopped 2024-04-15T15:21:29.424 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-15T15:21:29.424 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.489 INFO:tasks.ceph.osd.7:Stopped 2024-04-15T15:21:29.489 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-15T15:21:29.490 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-15T15:21:29.490 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.568 INFO:tasks.ceph.mgr.y:Stopped 2024-04-15T15:21:29.568 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-15T15:21:29.569 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.662 INFO:tasks.ceph.mgr.x:Stopped 2024-04-15T15:21:29.662 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-15T15:21:29.662 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-15T15:21:29.662 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.712 INFO:tasks.ceph.mon.a:Stopped 2024-04-15T15:21:29.712 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-15T15:21:29.712 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.744 INFO:tasks.ceph.mon.b:Stopped 2024-04-15T15:21:29.744 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-15T15:21:29.744 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:21:29.809 INFO:tasks.ceph.mon.c:Stopped 2024-04-15T15:21:29.810 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-15T15:21:29.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow request' | head -n 1 2024-04-15T15:21:29.931 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:21:29.931 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-15T15:21:30.174 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:21:30.174 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-15T15:21:30.358 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:21:30.358 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-15T15:21:30.516 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:21:30.517 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-15T15:21:30.679 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:21:30.679 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-15T15:21:30.936 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:21:30.937 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-15T15:21:31.072 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:21:31.073 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-15T15:21:31.237 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:21:31.237 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-15T15:21:31.384 INFO:tasks.ceph:Archiving mon data... 2024-04-15T15:21:31.386 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/data/mon.a.tgz 2024-04-15T15:21:31.387 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-04-15T15:21:31.450 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.kiwJcGLkML 2024-04-15T15:21:31.450 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.kiwJcGLkML 2024-04-15T15:21:31.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.kiwJcGLkML 2024-04-15T15:21:31.817 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.kiwJcGLkML is 344KB 2024-04-15T15:21:31.930 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.kiwJcGLkML 2024-04-15T15:21:31.990 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/data/mon.b.tgz 2024-04-15T15:21:31.990 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-04-15T15:21:32.046 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.nvlIFGHklP 2024-04-15T15:21:32.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nvlIFGHklP 2024-04-15T15:21:32.290 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.nvlIFGHklP 2024-04-15T15:21:32.445 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.nvlIFGHklP is 467KB 2024-04-15T15:21:32.558 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.nvlIFGHklP 2024-04-15T15:21:32.615 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/data/mon.c.tgz 2024-04-15T15:21:32.615 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-04-15T15:21:32.666 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.flSZJIWNQC 2024-04-15T15:21:32.666 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.flSZJIWNQC 2024-04-15T15:21:32.853 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.flSZJIWNQC 2024-04-15T15:21:32.991 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.flSZJIWNQC is 467KB 2024-04-15T15:21:33.100 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.flSZJIWNQC 2024-04-15T15:21:33.156 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-15T15:21:33.157 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-15T15:21:33.165 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-15T15:21:33.317 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-15T15:21:33.374 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-15T15:21:33.375 INFO:tasks.ceph:Archiving crash dumps... 2024-04-15T15:21:33.377 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/remote/smithi049/crash 2024-04-15T15:21:33.377 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-15T15:21:33.486 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/remote/smithi107/crash 2024-04-15T15:21:33.487 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-15T15:21:33.587 INFO:tasks.ceph:Shutting down logrotate 2024-04-15T15:21:33.587 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:33.595 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:33.752 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:33.864 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-15T15:21:33.960 INFO:tasks.ceph:Compressing logs... 2024-04-15T15:21:33.961 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-15T15:21:33.969 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-15T15:21:34.053 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37102.log 2024-04-15T15:21:34.054 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42311.log 2024-04-15T15:21:34.055 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.37102.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-15T15:21:34.055 INFO:teuthology.orchestra.run.smithi107.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37102.log.gz 2024-04-15T15:21:34.056 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.1.42311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37182.log 2024-04-15T15:21:34.056 INFO:teuthology.orchestra.run.smithi107.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42311.log.gz 2024-04-15T15:21:34.057 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37262.log 2024-04-15T15:21:34.059 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.37182.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37182.log.gz 2024-04-15T15:21:34.059 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43328.log 2024-04-15T15:21:34.060 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.37262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43324.log 2024-04-15T15:21:34.060 INFO:teuthology.orchestra.run.smithi107.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37262.log.gz 2024-04-15T15:21:34.061 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.1.43328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43328.log.gz 2024-04-15T15:21:34.061 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43653.log 2024-04-15T15:21:34.062 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.1.43324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43324.log.gz 2024-04-15T15:21:34.062 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43865.log 2024-04-15T15:21:34.063 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.1.43653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43653.log.gz 2024-04-15T15:21:34.063 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37222.log 2024-04-15T15:21:34.064 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.3.43865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43865.log.gz 2024-04-15T15:21:34.064 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43869.log 2024-04-15T15:21:34.065 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51087.log 2024-04-15T15:21:34.066 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.37222.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-15T15:21:34.066 INFO:teuthology.orchestra.run.smithi107.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37222.log.gz 2024-04-15T15:21:34.067 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.3.43869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43869.log.gz 2024-04-15T15:21:34.068 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-15T15:21:34.068 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-15T15:21:34.068 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42849.log 2024-04-15T15:21:34.068 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42176.log 2024-04-15T15:21:34.069 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51087.log.gz 2024-04-15T15:21:34.071 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42849.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-15T15:21:34.071 INFO:teuthology.orchestra.run.smithi049.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42849.log.gz 2024-04-15T15:21:34.071 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47544.log 2024-04-15T15:21:34.071 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42176.log.gz 2024-04-15T15:21:34.072 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41965.log 2024-04-15T15:21:34.073 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42710.log 2024-04-15T15:21:34.074 INFO:teuthology.orchestra.run.smithi107.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-15T15:21:34.074 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37062.log 2024-04-15T15:21:34.074 INFO:teuthology.orchestra.run.smithi107.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-15T15:21:34.074 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.3.42710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42710.log.gz 2024-04-15T15:21:34.074 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-15T15:21:34.074 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.37062.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-15T15:21:34.074 INFO:teuthology.orchestra.run.smithi107.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37062.log.gz 2024-04-15T15:21:34.075 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37142.log 2024-04-15T15:21:34.076 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42292.log 2024-04-15T15:21:34.076 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.37142.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-15T15:21:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46232.log 2024-04-15T15:21:34.078 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47544.log.gz 2024-04-15T15:21:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51477.log 2024-04-15T15:21:34.078 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41965.log.gz 2024-04-15T15:21:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42519.log 2024-04-15T15:21:34.078 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46232.log.gz 2024-04-15T15:21:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52217.log 2024-04-15T15:21:34.079 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51477.log.gz 2024-04-15T15:21:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42519.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42519.log.gz 2024-04-15T15:21:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52530.log 2024-04-15T15:21:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51881.log 2024-04-15T15:21:34.081 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52217.log.gz 2024-04-15T15:21:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52530.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52530.log.gz 2024-04-15T15:21:34.082 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47882.log 2024-04-15T15:21:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53770.log 2024-04-15T15:21:34.082 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51881.log.gz 2024-04-15T15:21:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41406.log 2024-04-15T15:21:34.082 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47882.log.gz 2024-04-15T15:21:34.083 INFO:teuthology.orchestra.run.smithi107.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37142.log.gz 2024-04-15T15:21:34.083 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.1.42292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36982.log 2024-04-15T15:21:34.083 INFO:teuthology.orchestra.run.smithi107.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42292.log.gz 2024-04-15T15:21:34.083 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43657.log 2024-04-15T15:21:34.083 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.36982.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-15T15:21:34.083 INFO:teuthology.orchestra.run.smithi107.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36982.log.gz 2024-04-15T15:21:34.084 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.1.43657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43657.log.gz 2024-04-15T15:21:34.084 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-15T15:21:34.085 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37022.log 2024-04-15T15:21:34.085 INFO:teuthology.orchestra.run.smithi107.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-15T15:21:34.085 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mon.b.log: 91.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-15T15:21:34.088 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42525.log 2024-04-15T15:21:34.088 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41406.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53770.log.gz 2024-04-15T15:21:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41406.log.gz -- 2024-04-15T15:21:34.089 INFO:teuthology.orchestra.run.smithi049.stderr: /var/log/ceph/ceph-client.admin.37030.log 2024-04-15T15:21:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45944.log 2024-04-15T15:21:34.089 INFO:teuthology.orchestra.run.smithi049.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42525.log.gz 2024-04-15T15:21:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.37030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52343.log 2024-04-15T15:21:34.090 INFO:teuthology.orchestra.run.smithi049.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.37030.log.gz 2024-04-15T15:21:34.090 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50884.log 2024-04-15T15:21:34.090 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45944.log.gz 2024-04-15T15:21:34.090 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47048.log 2024-04-15T15:21:34.093 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.37022.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37022.log.gz 2024-04-15T15:21:34.093 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-15T15:21:34.093 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42706.log 2024-04-15T15:21:34.093 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-15T15:21:34.093 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.3.42706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42706.log.gz 2024-04-15T15:21:34.095 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50884.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52343.log.gz 2024-04-15T15:21:34.095 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43477.log 2024-04-15T15:21:34.095 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50884.log.gz 2024-04-15T15:21:34.095 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45986.log 2024-04-15T15:21:34.095 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47048.log.gz 2024-04-15T15:21:34.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48806.log 2024-04-15T15:21:34.096 INFO:teuthology.orchestra.run.smithi049.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43477.log.gz 2024-04-15T15:21:34.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.50129.log 2024-04-15T15:21:34.096 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45986.log.gz 2024-04-15T15:21:34.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.48806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48806.log.gz 2024-04-15T15:21:34.096 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46880.log 2024-04-15T15:21:34.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.2.50129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.50129.log.gz 2024-04-15T15:21:34.096 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48278.log 2024-04-15T15:21:34.098 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46880.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46880.log.gz 2024-04-15T15:21:34.098 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51129.log 2024-04-15T15:21:34.098 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52007.log 2024-04-15T15:21:34.098 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51129.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48278.log.gz 2024-04-15T15:21:34.098 INFO:teuthology.orchestra.run.smithi049.stderr:gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51129.log.gz 2024-04-15T15:21:34.098 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45371.log 2024-04-15T15:21:34.099 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43603.log 2024-04-15T15:21:34.099 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52007.log.gz 2024-04-15T15:21:34.100 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52091.log 2024-04-15T15:21:34.100 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45371.log.gz 2024-04-15T15:21:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43603.log: gzip -5 88.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.43603.log.gz -- 2024-04-15T15:21:34.101 INFO:teuthology.orchestra.run.smithi049.stderr: /var/log/ceph/ceph-client.admin.45413.log 2024-04-15T15:21:34.102 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47384.log 2024-04-15T15:21:34.103 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52091.log.gz 2024-04-15T15:21:34.103 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50742.log 2024-04-15T15:21:34.104 INFO:teuthology.orchestra.run.smithi049.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45413.log.gz 2024-04-15T15:21:34.105 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47384.log: gzip -5 88.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47384.log.gz 2024-04-15T15:21:34.105 INFO:teuthology.orchestra.run.smithi049.stderr: -- /var/log/ceph/ceph-client.admin.43561.log 2024-04-15T15:21:34.106 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49236.log 2024-04-15T15:21:34.106 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50742.log.gz 2024-04-15T15:21:34.107 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42511.log 2024-04-15T15:21:34.107 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43561.log.gz 2024-04-15T15:21:34.108 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49236.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49236.log.gzgzip 2024-04-15T15:21:34.108 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44783.log 2024-04-15T15:21:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42858.log 2024-04-15T15:21:34.110 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42511.log.gz 2024-04-15T15:21:34.111 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47966.log 2024-04-15T15:21:34.111 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44783.log.gz 2024-04-15T15:21:34.112 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42858.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42858.log.gz 2024-04-15T15:21:34.112 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53014.log 2024-04-15T15:21:34.114 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53098.log 2024-04-15T15:21:34.114 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47966.log.gz 2024-04-15T15:21:34.114 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50191.log 2024-04-15T15:21:34.115 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53014.log.gz 2024-04-15T15:21:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46466.log 2024-04-15T15:21:34.116 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53098.log.gz 2024-04-15T15:21:34.117 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46922.log 2024-04-15T15:21:34.117 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50191.log.gz 2024-04-15T15:21:34.118 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52972.log 2024-04-15T15:21:34.119 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46466.log.gz 2024-04-15T15:21:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46922.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.46922.log.gz 2024-04-15T15:21:34.120 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53602.log 2024-04-15T15:21:34.121 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53870.log 2024-04-15T15:21:34.121 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52972.log.gz 2024-04-15T15:21:34.122 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45161.log 2024-04-15T15:21:34.122 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53602.log.gz 2024-04-15T15:21:34.123 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53870.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53870.log.gz 2024-04-15T15:21:34.123 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50796.log 2024-04-15T15:21:34.125 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53912.log 2024-04-15T15:21:34.125 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45161.log.gz 2024-04-15T15:21:34.126 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44650.log 2024-04-15T15:21:34.126 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50796.log.gz 2024-04-15T15:21:34.127 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53912.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.53912.log.gz 2024-04-15T15:21:34.127 INFO:teuthology.orchestra.run.smithi049.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46754.log 2024-04-15T15:21:34.128 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.2.44650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44650.log.gz 2024-04-15T15:21:34.128 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52427.log 2024-04-15T15:21:34.129 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41238.log 2024-04-15T15:21:34.129 INFO:teuthology.orchestra.run.smithi049.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46754.log.gz 2024-04-15T15:21:34.130 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51839.log 2024-04-15T15:21:34.130 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52427.log.gz 2024-04-15T15:21:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47586.log 2024-04-15T15:21:34.132 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41238.log.gz 2024-04-15T15:21:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48148.log 2024-04-15T15:21:34.133 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51839.log.gz 2024-04-15T15:21:34.134 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47586.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47586.log.gz -5 2024-04-15T15:21:34.134 INFO:teuthology.orchestra.run.smithi049.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42187.log 2024-04-15T15:21:34.135 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50396.log 2024-04-15T15:21:34.135 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48148.log.gz 2024-04-15T15:21:34.136 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52049.log 2024-04-15T15:21:34.136 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42187.log.gz 2024-04-15T15:21:34.137 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49895.log 2024-04-15T15:21:34.138 INFO:teuthology.orchestra.run.smithi049.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50396.log.gz 2024-04-15T15:21:34.138 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51965.log 2024-04-15T15:21:34.139 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52049.log.gz 2024-04-15T15:21:34.140 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49566.log 2024-04-15T15:21:34.140 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51965.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49895.log.gz 2024-04-15T15:21:34.141 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51965.log.gz 2024-04-15T15:21:34.141 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42520.log 2024-04-15T15:21:34.142 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42188.log 2024-04-15T15:21:34.142 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7%/var/log/ceph/ceph-client.admin.42520.log: -- replaced with /var/log/ceph/ceph-client.admin.49566.log.gz 2024-04-15T15:21:34.143 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49194.log 2024-04-15T15:21:34.143 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42520.log.gz 2024-04-15T15:21:34.144 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41713.log 2024-04-15T15:21:34.145 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42188.log.gz 2024-04-15T15:21:34.145 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51677.log 2024-04-15T15:21:34.146 INFO:teuthology.orchestra.run.smithi049.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49194.log.gz 2024-04-15T15:21:34.146 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41713.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41713.log.gz 2024-04-15T15:21:34.147 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53056.log 2024-04-15T15:21:34.148 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53560.log 2024-04-15T15:21:34.148 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51677.log.gz 2024-04-15T15:21:34.149 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48488.log 2024-04-15T15:21:34.150 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53056.log.gz 2024-04-15T15:21:34.150 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mon.c.log: 93.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-15T15:21:34.151 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46106.log 2024-04-15T15:21:34.151 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53560.log.gz 2024-04-15T15:21:34.152 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41839.log 2024-04-15T15:21:34.152 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48488.log.gz 2024-04-15T15:21:34.153 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43825.log 2024-04-15T15:21:34.153 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46106.log.gz 2024-04-15T15:21:34.154 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51923.log 2024-04-15T15:21:34.154 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41839.log.gz 2024-04-15T15:21:34.156 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43645.log 2024-04-15T15:21:34.156 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43825.log.gz 2024-04-15T15:21:34.157 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52656.log 2024-04-15T15:21:34.157 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51923.log.gz 2024-04-15T15:21:34.158 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43645.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43645.log.gzgzip 2024-04-15T15:21:34.158 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45707.log 2024-04-15T15:21:34.159 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47258.log 2024-04-15T15:21:34.160 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52656.log.gz 2024-04-15T15:21:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45707.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31016.log 2024-04-15T15:21:34.160 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45707.log.gz 2024-04-15T15:21:34.161 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47258.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.47258.log.gz 2024-04-15T15:21:34.161 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47756.log 2024-04-15T15:21:34.163 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.tmp-client.admin.31016.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31016.log.gz 2024-04-15T15:21:34.163 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46670.log 2024-04-15T15:21:34.163 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37287.log 2024-04-15T15:21:34.164 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47756.log.gz 2024-04-15T15:21:34.165 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46670.log: gzip 88.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.46670.log.gz --verbose 2024-04-15T15:21:34.165 INFO:teuthology.orchestra.run.smithi049.stderr: -- /var/log/ceph/ceph-client.admin.46424.log 2024-04-15T15:21:34.166 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.37287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47924.log 2024-04-15T15:21:34.166 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37287.log.gz 2024-04-15T15:21:34.167 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47216.log 2024-04-15T15:21:34.167 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46424.log.gz 2024-04-15T15:21:34.168 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47924.log: gzip 89.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.47924.log.gz 2024-04-15T15:21:34.168 INFO:teuthology.orchestra.run.smithi049.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41671.log 2024-04-15T15:21:34.170 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46796.log 2024-04-15T15:21:34.170 INFO:teuthology.orchestra.run.smithi049.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47216.log.gz 2024-04-15T15:21:34.171 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45539.log 2024-04-15T15:21:34.171 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41671.log.gz 2024-04-15T15:21:34.172 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46796.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-15T15:21:34.172 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46796.log.gz 2024-04-15T15:21:34.173 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47132.log 2024-04-15T15:21:34.174 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45539.log.gz 2024-04-15T15:21:34.174 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41501.log 2024-04-15T15:21:34.175 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47132.log: gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.47132.log.gz 2024-04-15T15:21:34.176 INFO:teuthology.orchestra.run.smithi049.stderr: /var/log/ceph/ceph-client.admin.48010.log 2024-04-15T15:21:34.176 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-15T15:21:34.177 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51045.log 2024-04-15T15:21:34.177 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48010.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41501.log.gz 2024-04-15T15:21:34.178 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48010.log.gz 2024-04-15T15:21:34.178 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41112.log 2024-04-15T15:21:34.179 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50682.log 2024-04-15T15:21:34.179 INFO:teuthology.orchestra.run.smithi049.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51045.log.gz 2024-04-15T15:21:34.180 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41112.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.41112.log.gz 2024-04-15T15:21:34.180 INFO:teuthology.orchestra.run.smithi049.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49320.log 2024-04-15T15:21:34.181 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41923.log 2024-04-15T15:21:34.182 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50682.log.gz 2024-04-15T15:21:34.182 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53308.log 2024-04-15T15:21:34.183 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49320.log.gz 2024-04-15T15:21:34.183 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41923.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.41923.log.gz -- 2024-04-15T15:21:34.184 INFO:teuthology.orchestra.run.smithi049.stderr: /var/log/ceph/ceph-client.admin.43393.log 2024-04-15T15:21:34.185 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43309.log 2024-04-15T15:21:34.185 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53308.log.gz 2024-04-15T15:21:34.186 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52301.log 2024-04-15T15:21:34.186 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43393.log.gz 2024-04-15T15:21:34.187 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41347.log 2024-04-15T15:21:34.187 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43309.log.gz 2024-04-15T15:21:34.188 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41587.log 2024-04-15T15:21:34.189 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52301.log.gz 2024-04-15T15:21:34.190 INFO:teuthology.orchestra.run.smithi107.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-15T15:21:34.190 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45860.log 2024-04-15T15:21:34.191 INFO:teuthology.orchestra.run.smithi049.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41347.log.gz 2024-04-15T15:21:34.191 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41545.log 2024-04-15T15:21:34.191 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41587.log.gz 2024-04-15T15:21:34.192 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51797.log 2024-04-15T15:21:34.192 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45860.log.gz 2024-04-15T15:21:34.193 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53350.log 2024-04-15T15:21:34.193 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41545.log.gz 2024-04-15T15:21:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47840.log 2024-04-15T15:21:34.194 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51797.log.gz 2024-04-15T15:21:34.195 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53140.log 2024-04-15T15:21:34.196 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53350.log.gz 2024-04-15T15:21:34.196 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47672.log 2024-04-15T15:21:34.197 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47840.log.gz 2024-04-15T15:21:34.198 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42856.log 2024-04-15T15:21:34.198 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53140.log.gz 2024-04-15T15:21:34.199 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47672.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47672.log.gz 2024-04-15T15:21:34.200 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45245.log 2024-04-15T15:21:34.200 INFO:teuthology.orchestra.run.smithi107.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-15T15:21:34.200 INFO:teuthology.orchestra.run.smithi107.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-15T15:21:34.201 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44867.log 2024-04-15T15:21:34.202 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42856.log.gz 2024-04-15T15:21:34.202 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51593.log 2024-04-15T15:21:34.202 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45245.log.gz 2024-04-15T15:21:34.203 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49022.log 2024-04-15T15:21:34.203 INFO:teuthology.orchestra.run.smithi049.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44867.log.gz 2024-04-15T15:21:34.204 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49937.log 2024-04-15T15:21:34.204 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51593.log.gz 2024-04-15T15:21:34.205 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52175.log 2024-04-15T15:21:34.206 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49022.log.gz 2024-04-15T15:21:34.207 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46190.log 2024-04-15T15:21:34.207 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49937.log.gz 2024-04-15T15:21:34.208 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47300.log 2024-04-15T15:21:34.208 INFO:teuthology.orchestra.run.smithi049.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52175.log.gz 2024-04-15T15:21:34.209 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51393.log 2024-04-15T15:21:34.209 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46190.log.gz 2024-04-15T15:21:34.210 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42175.log 2024-04-15T15:21:34.211 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47300.log.gz 2024-04-15T15:21:34.211 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47714.log 2024-04-15T15:21:34.212 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51393.log.gz 2024-04-15T15:21:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42175.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-15T15:21:34.213 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42175.log.gz 2024-04-15T15:21:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46382.log 2024-04-15T15:21:34.214 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47714.log.gz 2024-04-15T15:21:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48054.log 2024-04-15T15:21:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52259.log 2024-04-15T15:21:34.217 INFO:teuthology.orchestra.run.smithi049.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46382.log.gz 2024-04-15T15:21:34.218 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48885.log 2024-04-15T15:21:34.218 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48054.log.gz 2024-04-15T15:21:34.219 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45497.log 2024-04-15T15:21:34.220 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52259.log.gz 2024-04-15T15:21:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43057.log 2024-04-15T15:21:34.221 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48885.log.gz 2024-04-15T15:21:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42180.log 2024-04-15T15:21:34.222 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45497.log.gz 2024-04-15T15:21:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43687.log 2024-04-15T15:21:34.224 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43057.log.gz 2024-04-15T15:21:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51307.log 2024-04-15T15:21:34.225 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42180.log.gz 2024-04-15T15:21:34.226 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46712.log 2024-04-15T15:21:34.227 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51307.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43687.log.gz 2024-04-15T15:21:34.228 INFO:teuthology.orchestra.run.smithi107.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-15T15:21:34.228 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51307.log.gz 2024-04-15T15:21:34.229 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48194.log 2024-04-15T15:21:34.229 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42517.log 2024-04-15T15:21:34.230 INFO:teuthology.orchestra.run.smithi107.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-15T15:21:34.230 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46712.log.gz 2024-04-15T15:21:34.231 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53224.log 2024-04-15T15:21:34.231 INFO:teuthology.orchestra.run.smithi049.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48194.log.gz 2024-04-15T15:21:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42517.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42517.log.gz 2024-04-15T15:21:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42855.log 2024-04-15T15:21:34.234 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51719.log 2024-04-15T15:21:34.235 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53224.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53224.log.gz 2024-04-15T15:21:34.235 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44208.log 2024-04-15T15:21:34.235 INFO:teuthology.orchestra.run.smithi107.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-15T15:21:34.236 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42855.log.gz 2024-04-15T15:21:34.236 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51719.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.42007.log 2024-04-15T15:21:34.236 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51719.log.gz 2024-04-15T15:21:34.237 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.44208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44208.log.gz 2024-04-15T15:21:34.237 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42819.log 2024-04-15T15:21:34.238 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-04-15T15:21:34.238 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m0.221s 2024-04-15T15:21:34.239 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m0.865s 2024-04-15T15:21:34.239 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m0.192s 2024-04-15T15:21:34.240 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46508.log 2024-04-15T15:21:34.240 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42007.log.gz 2024-04-15T15:21:34.240 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42514.log 2024-04-15T15:21:34.241 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42819.log.gz 2024-04-15T15:21:34.241 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41755.log 2024-04-15T15:21:34.242 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46508.log.gz 2024-04-15T15:21:34.242 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41193.log 2024-04-15T15:21:34.243 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42514.log.gz 2024-04-15T15:21:34.244 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53266.log 2024-04-15T15:21:34.244 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41755.log.gz 2024-04-15T15:21:34.245 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46274.log 2024-04-15T15:21:34.245 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41193.log.gz 2024-04-15T15:21:34.246 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47006.log 2024-04-15T15:21:34.246 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53266.log.gz 2024-04-15T15:21:34.247 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46274.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-15T15:21:34.248 INFO:teuthology.orchestra.run.smithi049.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46274.log.gz 2024-04-15T15:21:34.248 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42521.log 2024-04-15T15:21:34.249 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47006.log.gz 2024-04-15T15:21:34.250 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45665.log 2024-04-15T15:21:34.251 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48596.log 2024-04-15T15:21:34.252 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42521.log: /var/log/ceph/ceph-client.admin.45665.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42521.log.gz 2024-04-15T15:21:34.252 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45665.log.gz 2024-04-15T15:21:34.252 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49829.log 2024-04-15T15:21:34.253 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44187.log 2024-04-15T15:21:34.254 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48596.log.gz 2024-04-15T15:21:34.254 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.49829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49829.log.gz 2024-04-15T15:21:34.255 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46148.log 2024-04-15T15:21:34.256 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.44187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50838.log 2024-04-15T15:21:34.256 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44187.log.gz 2024-04-15T15:21:34.257 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48446.log 2024-04-15T15:21:34.257 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46148.log.gz 2024-04-15T15:21:34.258 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44951.log 2024-04-15T15:21:34.259 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50838.log.gz 2024-04-15T15:21:34.260 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48446.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48446.log.gz 2024-04-15T15:21:34.260 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49833.log 2024-04-15T15:21:34.261 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42845.log 2024-04-15T15:21:34.262 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44951.log: /var/log/ceph/ceph-client.0.49833.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44951.log.gz 2024-04-15T15:21:34.262 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49833.log.gz 2024-04-15T15:21:34.263 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51351.log 2024-04-15T15:21:34.264 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42845.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42845.log.gz 2024-04-15T15:21:34.264 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44909.log 2024-04-15T15:21:34.265 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41016.log 2024-04-15T15:21:34.266 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44909.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51351.log.gz 2024-04-15T15:21:34.267 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51635.log 2024-04-15T15:21:34.267 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44909.log.gz 2024-04-15T15:21:34.268 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45581.log 2024-04-15T15:21:34.268 INFO:teuthology.orchestra.run.smithi049.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41016.log.gz 2024-04-15T15:21:34.269 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52698.log 2024-04-15T15:21:34.270 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51635.log.gz 2024-04-15T15:21:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48802.log 2024-04-15T15:21:34.271 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45581.log.gz 2024-04-15T15:21:34.272 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43267.log 2024-04-15T15:21:34.272 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52698.log.gz 2024-04-15T15:21:34.273 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.48802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49524.log 2024-04-15T15:21:34.273 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48802.log.gz 2024-04-15T15:21:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42861.log 2024-04-15T15:21:34.275 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43267.log.gz 2024-04-15T15:21:34.276 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48362.log 2024-04-15T15:21:34.276 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49524.log.gz 2024-04-15T15:21:34.277 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53434.log 2024-04-15T15:21:34.277 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42861.log.gz 2024-04-15T15:21:34.278 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49078.log 2024-04-15T15:21:34.279 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48362.log.gz 2024-04-15T15:21:34.280 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-04-15T15:21:34.280 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53434.log.gz 2024-04-15T15:21:34.281 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46628.log 2024-04-15T15:21:34.281 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49078.log.gz 2024-04-15T15:21:34.282 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41449.log 2024-04-15T15:21:34.283 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-04-15T15:21:34.284 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50926.log 2024-04-15T15:21:34.284 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46628.log.gz 2024-04-15T15:21:34.285 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48980.log 2024-04-15T15:21:34.285 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41449.log.gz 2024-04-15T15:21:34.286 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46550.log 2024-04-15T15:21:34.287 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50926.log.gz 2024-04-15T15:21:34.288 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48530.log 2024-04-15T15:21:34.288 INFO:teuthology.orchestra.run.smithi049.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48980.log.gz 2024-04-15T15:21:34.289 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45077.log 2024-04-15T15:21:34.289 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46550.log.gz 2024-04-15T15:21:34.290 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48530.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-15T15:21:34.291 INFO:teuthology.orchestra.run.smithi049.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48530.log.gz 2024-04-15T15:21:34.292 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2024-04-15T15:21:34.292 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45077.log.gz 2024-04-15T15:21:34.293 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47464.log 2024-04-15T15:21:34.294 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48320.log 2024-04-15T15:21:34.295 INFO:teuthology.orchestra.run.smithi049.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2024-04-15T15:21:34.296 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50334.log 2024-04-15T15:21:34.296 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47464.log.gz 2024-04-15T15:21:34.297 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52614.log 2024-04-15T15:21:34.298 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48320.log.gz 2024-04-15T15:21:34.299 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47342.log 2024-04-15T15:21:34.299 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50334.log.gz 2024-04-15T15:21:34.300 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45203.log 2024-04-15T15:21:34.300 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52614.log.gz 2024-04-15T15:21:34.301 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46028.log 2024-04-15T15:21:34.301 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47342.log.gz 2024-04-15T15:21:34.302 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46838.log 2024-04-15T15:21:34.303 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45203.log.gz 2024-04-15T15:21:34.303 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47090.log 2024-04-15T15:21:34.304 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46028.log.gz 2024-04-15T15:21:34.305 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52133.log 2024-04-15T15:21:34.305 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46838.log.gz 2024-04-15T15:21:34.306 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49398.log 2024-04-15T15:21:34.307 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47090.log.gz 2024-04-15T15:21:34.307 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53644.log 2024-04-15T15:21:34.308 INFO:teuthology.orchestra.run.smithi049.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52133.log.gz 2024-04-15T15:21:34.309 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51551.log 2024-04-15T15:21:34.309 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49398.log.gz 2024-04-15T15:21:34.310 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53644.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.53728.log 2024-04-15T15:21:34.310 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53644.log.gz 2024-04-15T15:21:34.311 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48404.log 2024-04-15T15:21:34.312 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51551.log.gz 2024-04-15T15:21:34.313 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53728.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53728.log.gz -5 2024-04-15T15:21:34.313 INFO:teuthology.orchestra.run.smithi049.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43141.log 2024-04-15T15:21:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52740.log 2024-04-15T15:21:34.314 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48404.log.gz 2024-04-15T15:21:34.315 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48106.log 2024-04-15T15:21:34.315 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43141.log.gz 2024-04-15T15:21:34.316 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48236.log 2024-04-15T15:21:34.317 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52740.log.gz 2024-04-15T15:21:34.317 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41629.log 2024-04-15T15:21:34.318 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48106.log.gz 2024-04-15T15:21:34.319 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48236.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-15T15:21:34.319 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48236.log.gz 2024-04-15T15:21:34.320 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41629.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45455.log 2024-04-15T15:21:34.320 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41629.log.gz 2024-04-15T15:21:34.321 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47174.log 2024-04-15T15:21:34.323 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41881.log 2024-04-15T15:21:34.323 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45455.log.gz 2024-04-15T15:21:34.324 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47174.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47174.log.gz 2024-04-15T15:21:34.325 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52572.log 2024-04-15T15:21:34.326 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41881.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41881.log.gz 2024-04-15T15:21:34.326 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43519.log 2024-04-15T15:21:34.327 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42174.log 2024-04-15T15:21:34.328 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52572.log.gz 2024-04-15T15:21:34.328 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43519.log: 90.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51171.log 2024-04-15T15:21:34.329 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-15T15:21:34.329 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43519.log.gz 2024-04-15T15:21:34.330 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45623.log 2024-04-15T15:21:34.330 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42174.log.gz 2024-04-15T15:21:34.331 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49440.log 2024-04-15T15:21:34.331 INFO:teuthology.orchestra.run.smithi049.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51171.log.gz 2024-04-15T15:21:34.332 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42853.log 2024-04-15T15:21:34.333 INFO:teuthology.orchestra.run.smithi049.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45623.log.gz 2024-04-15T15:21:34.333 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49440.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-15T15:21:34.334 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49440.log.gz 2024-04-15T15:21:34.334 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45287.log 2024-04-15T15:21:34.335 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42853.log.gz 2024-04-15T15:21:34.335 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43351.log 2024-04-15T15:21:34.336 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45287.log: 85.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-15T15:21:34.337 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49152.log 2024-04-15T15:21:34.337 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45287.log.gz 2024-04-15T15:21:34.338 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42186.log 2024-04-15T15:21:34.338 INFO:teuthology.orchestra.run.smithi049.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43351.log.gz 2024-04-15T15:21:34.339 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44646.log 2024-04-15T15:21:34.340 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49152.log.gz 2024-04-15T15:21:34.340 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42186.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42186.log.gz 2024-04-15T15:21:34.340 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49278.log 2024-04-15T15:21:34.341 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.2.44646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44646.log.gz 2024-04-15T15:21:34.342 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42189.log 2024-04-15T15:21:34.342 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49278.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-15T15:21:34.343 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49278.log.gz 2024-04-15T15:21:34.343 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45329.log 2024-04-15T15:21:34.344 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42189.log.gz 2024-04-15T15:21:34.345 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-15T15:21:34.346 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52385.log 2024-04-15T15:21:34.346 INFO:teuthology.orchestra.run.smithi049.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45329.log.gz 2024-04-15T15:21:34.347 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53518.log 2024-04-15T15:21:34.349 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52385.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52385.log.gz 2024-04-15T15:21:34.350 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.50125.log 2024-04-15T15:21:34.350 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53518.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53518.log.gz 2024-04-15T15:21:34.351 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52487.log 2024-04-15T15:21:34.352 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.2.50125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.50125.log.gz 2024-04-15T15:21:34.353 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44825.log 2024-04-15T15:21:34.355 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52487.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52487.log.gz 2024-04-15T15:21:34.356 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42510.log/var/log/ceph/ceph-client.admin.44825.log: 2024-04-15T15:21:34.357 INFO:teuthology.orchestra.run.smithi049.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44825.log.gz 2024-04-15T15:21:34.358 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43099.log 2024-04-15T15:21:34.359 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42510.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42510.log.gz 2024-04-15T15:21:34.360 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41284.log 2024-04-15T15:21:34.361 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43099.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43099.log.gz 2024-04-15T15:21:34.362 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45119.log 2024-04-15T15:21:34.364 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41284.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-15T15:21:34.364 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41284.log.gz 2024-04-15T15:21:34.364 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43225.log 2024-04-15T15:21:34.365 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45119.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45119.log.gz 2024-04-15T15:21:34.365 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47798.log 2024-04-15T15:21:34.366 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5/var/log/ceph/ceph-client.admin.43225.log: --verbose -- /var/log/ceph/ceph-client.admin.53182.log 2024-04-15T15:21:34.367 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43225.log.gz 2024-04-15T15:21:34.368 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51213.log 2024-04-15T15:21:34.368 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47798.log.gz 2024-04-15T15:21:34.369 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48938.log 2024-04-15T15:21:34.369 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53182.log.gz 2024-04-15T15:21:34.370 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45902.log 2024-04-15T15:21:34.371 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51213.log.gz 2024-04-15T15:21:34.372 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51435.log 2024-04-15T15:21:34.372 INFO:teuthology.orchestra.run.smithi049.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48938.log.gz 2024-04-15T15:21:34.373 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45902.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45902.log.gz 2024-04-15T15:21:34.373 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44342.log 2024-04-15T15:21:34.374 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52930.log 2024-04-15T15:21:34.375 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51435.log.gz 2024-04-15T15:21:34.376 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43435.log 2024-04-15T15:21:34.376 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44342.log.gz 2024-04-15T15:21:34.377 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52930.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52930.log.gz 2024-04-15T15:21:34.378 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45749.log 2024-04-15T15:21:34.381 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37399.log 2024-04-15T15:21:34.381 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43435.log.gz 2024-04-15T15:21:34.382 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53392.log 2024-04-15T15:21:34.382 INFO:teuthology.orchestra.run.smithi049.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45749.log.gz 2024-04-15T15:21:34.382 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.37399.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37399.log.gz 2024-04-15T15:21:34.383 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49641.log 2024-04-15T15:21:34.384 INFO:teuthology.orchestra.run.smithi049.stderr:gzip/var/log/ceph/ceph-client.admin.53392.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46964.log 2024-04-15T15:21:34.384 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53392.log.gz 2024-04-15T15:21:34.385 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49641.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49641.log.gz 2024-04-15T15:21:34.385 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44993.log 2024-04-15T15:21:34.385 INFO:teuthology.orchestra.run.smithi049.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-15T15:21:34.386 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47630.log 2024-04-15T15:21:34.386 INFO:teuthology.orchestra.run.smithi049.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46964.log.gz 2024-04-15T15:21:34.387 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44993.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44993.log.gz 2024-04-15T15:21:34.387 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49482.log 2024-04-15T15:21:34.388 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41797.log 2024-04-15T15:21:34.388 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47630.log.gz 2024-04-15T15:21:34.390 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53830.log 2024-04-15T15:21:34.390 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49482.log.gz 2024-04-15T15:21:34.390 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41797.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-15T15:21:34.391 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41797.log.gz 2024-04-15T15:21:34.392 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53830.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.53830.log.gz --verbose 2024-04-15T15:21:34.392 INFO:teuthology.orchestra.run.smithi049.stderr: -- /var/log/ceph/ceph-client.admin.53686.log 2024-04-15T15:21:34.393 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45035.log 2024-04-15T15:21:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53686.log: 91.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-15T15:21:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53476.log 2024-04-15T15:21:34.394 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53686.log.gz 2024-04-15T15:21:34.396 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45035.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45035.log.gz 2024-04-15T15:21:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53476.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53476.log.gz 2024-04-15T15:21:34.447 INFO:teuthology.orchestra.run.smithi049.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-15T15:21:34.467 INFO:teuthology.orchestra.run.smithi049.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-15T15:21:34.469 INFO:teuthology.orchestra.run.smithi049.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-15T15:21:34.477 INFO:teuthology.orchestra.run.smithi049.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-15T15:21:34.480 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-04-15T15:21:34.481 INFO:teuthology.orchestra.run.smithi049.stderr:real 0m0.462s 2024-04-15T15:21:34.481 INFO:teuthology.orchestra.run.smithi049.stderr:user 0m1.157s 2024-04-15T15:21:34.481 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m0.838s 2024-04-15T15:21:34.482 INFO:tasks.ceph:Archiving logs... 2024-04-15T15:21:34.482 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/remote/smithi049/log 2024-04-15T15:21:34.484 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-15T15:21:34.893 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/remote/smithi107/log 2024-04-15T15:21:34.894 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-15T15:21:35.117 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-15T15:21:35.144 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-15T15:21:35.145 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-15T15:21:35.153 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-15T15:21:35.307 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-15T15:21:35.316 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-15T15:21:35.316 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:21:35.325 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:21:35.376 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:21:35.379 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:21:35.387 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:21:35.387 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-04-15T15:21:35.387 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +1585us[+1585us] +/- 53ms 2024-04-15T15:21:35.387 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -1781us[-1781us] +/- 52ms 2024-04-15T15:21:35.387 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -675us[ -675us] +/- 77ms 2024-04-15T15:21:35.387 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:21:35.389 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:21:35.389 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-04-15T15:21:35.389 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 +1594us[+1591us] +/- 53ms 2024-04-15T15:21:35.390 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 -1768us[-1768us] +/- 52ms 2024-04-15T15:21:35.390 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -649us[ -649us] +/- 77ms 2024-04-15T15:21:35.390 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:21:35.391 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-15T15:21:35.403 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-15T15:21:35.403 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-15T15:21:35.419 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-15T15:21:35.670 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-15T15:21:35.886 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:21:36.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:21:36.004 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:21:36.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:21:36.109 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-15T15:21:36.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-15T15:21:36.137 INFO:teuthology.task.internal:Duration was 749.316562 seconds 2024-04-15T15:21:36.138 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-15T15:21:36.147 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-15T15:21:36.147 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-15T15:21:36.155 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-15T15:21:36.311 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:21:36.343 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:21:36.825 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-15T15:21:36.826 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-04-15T15:21:36.826 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-15T15:21:36.893 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-04-15T15:21:36.893 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-15T15:21:36.970 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-15T15:21:36.970 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-15T15:21:36.978 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-15T15:21:37.081 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-15T15:21:37.082 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-15T15:21:37.089 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-15T15:21:37.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-15T15:21:37.301 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-15T15:21:37.302 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-15T15:21:37.412 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-15T15:21:37.513 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-15T15:21:37.525 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:21:37.533 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:21:37.641 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-04-15T15:21:37.649 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-04-15T15:21:37.710 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:21:37.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:21:37.763 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:21:37.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:21:37.818 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-15T15:21:37.829 INFO:teuthology.task.internal:Transferring archived files... 2024-04-15T15:21:37.829 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/remote/smithi049 2024-04-15T15:21:37.830 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-15T15:21:37.950 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119/remote/smithi107 2024-04-15T15:21:37.950 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-15T15:21:38.057 INFO:teuthology.task.internal:Removing archive directory... 2024-04-15T15:21:38.058 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-15T15:21:38.066 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-15T15:21:38.124 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-15T15:21:38.135 INFO:teuthology.task.internal:Not uploading archives. 2024-04-15T15:21:38.135 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-15T15:21:38.150 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-15T15:21:38.151 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-15T15:21:38.158 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-15T15:21:38.212 INFO:teuthology.orchestra.run.smithi049.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 15 15:21 /home/ubuntu/cephtest 2024-04-15T15:21:38.214 INFO:teuthology.orchestra.run.smithi107.stdout: 395174 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 15 15:21 /home/ubuntu/cephtest 2024-04-15T15:21:38.217 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-15T15:21:38.230 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-15T15:21:38.261 INFO:teuthology.nuke:Checking targets against current locks 2024-04-15T15:21:38.290 DEBUG:teuthology.nuke:shortname: smithi049 2024-04-15T15:21:38.290 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-15T15:21:38.312 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 14:56:56.578553', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDB2/YdZYsSxPRvJ82wegIo13jQ1h3vGBQsXIodUm9Dh1Wb3P75Jv3mchNNDhv2hcIMgigAsLdPTiTAdgMChKRA='} 2024-04-15T15:21:38.316 DEBUG:teuthology.nuke:shortname: smithi107 2024-04-15T15:21:38.316 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-15T15:21:38.341 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:46:31-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657119', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 14:56:56.577236', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJQqSxuRx5wTNkPdbKM8TVcVuTeoPB/CT9UuQMYOsso+CKgHzs0p5W4GWzB0DfADwhDMn8308WtVk0DNgnQRUI='} 2024-04-15T15:21:38.367 INFO:teuthology.orchestra.console.smithi049:Power off 2024-04-15T15:21:38.368 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-15T15:21:38.394 DEBUG:teuthology.orchestra.console.smithi049:power off output: Chassis Power Control: Down/Off 2024-04-15T15:21:38.394 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:21:38.415 INFO:teuthology.orchestra.console.smithi107:Power off 2024-04-15T15:21:38.415 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-15T15:21:38.426 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-04-15T15:21:38.438 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2024-04-15T15:21:38.438 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:21:38.461 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-04-15T15:21:42.427 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:21:42.462 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:21:42.539 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-04-15T15:21:42.573 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-04-15T15:21:46.542 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:21:46.574 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:21:46.656 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is off 2024-04-15T15:21:46.685 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-04-15T15:21:46.758 INFO:teuthology.orchestra.console.smithi049:Power off completed 2024-04-15T15:21:50.689 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:21:50.799 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-04-15T15:21:50.900 INFO:teuthology.orchestra.console.smithi107:Power off completed 2024-04-15T15:21:51.005 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 client.1-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/quiesce} duration: 749.3165619373322 failure_reason: 'Test failure: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce)' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=3c6838482d344462b31d935fb75c8cb5 status: fail success: false 2024-04-15T15:21:51.005 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-15T15:21:51.093 INFO:teuthology.run:FAIL